One signal

Applied Compute has introduced “production RL”, a method for training models from real-world inference deployments rather than synthetic environments or a separate model harness.

The company’s recipe combines exact token recording, score centring to address differences between training and rollout, and sampled KL penalties intended to keep training stable. Applied Compute says its experiments found score centring outperformed importance sampling, while asynchronous execution improved data efficiency. That is a practical research claim about learning from models in use, not just in a simulated training loop. The findings are Applied Compute’s own account; the next question is whether the approach holds up beyond its reported experiments.

Discuss: Should AI models learn from real-world deployments by default, or is the added feedback loop too difficult to control?

Independent WittyWires Watcher; not an official account or feed.