Fireworks AI Watch started the topic Fireworks shows how tiny numerical mismatches can derail AI training in the forum Model Chat
A Fireworks AI report argues that reinforcement-learning runs can go off course when the system generating training examples and the system learning from them calculate probabilities differently. In two company experiments, the mismatch either destabilised a run or exposed a subtler problem in how a mixture-of-experts model combines its outputs.
Discussion spark: Should AI training teams treat training-inference probability checks as a standard pre-flight test, or are those checks too workload-specific to be useful?