Sakana AI trains a model on its own team’s research work

Sakana AI says its Multi-Agent Self-Supervision method improved a model’s results on four research benchmarks by having a team of virtual agents solve tasks, evaluate possible workflows and pass the strongest team’s experience back into training. The result is a concrete experiment in recursive AI improvement, though a two-cycle test on synthetic tasks is not a general proof that models can improve themselves safely or reliably.

Sakana AI Watch analysis

What happened

In a paper and accompanying project post, Sakana AI describes using one shared 27-billion-parameter open-weights model to play several roles. Virtual subagents tackle open-ended research tasks; the system searches for better ways to organise the team, judges those approaches and trains the shared model on the best team’s work.

After two cycles, Sakana AI reports that score per output token reached 1.2 to 1.6 times the base model’s level across four research benchmarks. The team says the work used synthetic open-ended research tasks. Those are the researchers’ reported results, not an independent replication.

Why it matters

The experiment tests a route to improvement that does not depend solely on humans supplying better examples. Instead, a model’s own team-generated work becomes training material for the shared model. Sakana AI also says the approach improved the capabilities of the system’s optimiser and evaluator, and that multi-agent trajectories provided more efficient training data than single-agent ones.

That makes the evaluator’s judgement a crucial part of the loop. If the model mistakes a polished error for progress, training may reinforce the very blind spot the process is meant to overcome. Sakana AI flags that risk itself; the results do not settle how well the method transfers beyond the tasks tested.

Our read

This is a serious, specific attempt at recursive self-improvement, not a grand claim that the machines have started grading their own homework and getting straight As. The interesting evidence is the reported benchmark gain; the interesting question is whether it holds up under independent testing and less synthetic work.

Sakana AI’s work gives readers a useful way to assess the next claim in this area: ask what the system improved, who judged that improvement and whether the result survives outside the loop that produced it.

What to watch

• Whether independent researchers reproduce the reported gains on all four benchmarks.
• How the method performs on real-world research tasks, not only synthetic ones.
• Whether evaluation can catch shared errors before they become training signals.

Discussion spark: Should a model be allowed to train on work its own agents judge successful, or should an independent evaluator be required before that work counts as progress?

not affiliated with or endorsed by Sakana AI