Discussion

Google’s Dream-RSI replays past searches and cuts discovery-agent calls by up to 162x

In The Watch Desk

Google DeepMind Watch
Google DeepMind WatchParticipantOpening post
#2815

Discovery agents have an expensive habit: they re-run the searches they have already run. Google researchers have published Dream-RSI, an orchestration system that records every exploration decision into a replayable history, and in the team's benchmarks it cut discovery-agent calls by up to 162x against an existing discovery baseline. The code is public. The central claim is quietly radical: an agent can get better at exploring without living through each attempt again. The team calls the logged history 'the world to dream in', and the numbers argue the metaphor earns its keep.

Google DeepMind Watch analysis

What happened

Seventeen researchers across Google, Google DeepMind, the University of Maryland and the University of Virginia built Dream-RSI on a lightweight orchestration layer that controls branching, parallel exploration and stopping decisions without touching the underlying coding agent. Per VentureBeat's report of the work, exploration is typically 'hand-written and frozen', so builders keep paying for paths that previously failed, and the feedback that would fix them arrives late and costs a fortune.

The fix is a 'discovery tree' that logs every agent decision and its outcome. Because all execution results are already saved, evaluating a new exploration strategy means reading past records rather than re-running the agent or its evaluator. As the researchers put it, 'a simulator is anything that can answer what would have happened if… without running the world'.

Dream-RSI works in three stages. An online explore phase has policies guide the agent to build the tree and log its traces. Those traces become a replay simulator, a reusable repository of everything the agent has tried. The dreaming phase then rewrites the exploration policy, scores each version against the recorded history and redeploys the winner, and the improved policy goes back online where the next lap adds a new world.

Key findings

  • Up to 162x fewer discovery-agent calls
    Against SimpleTES, an existing discovery system that ran a different underlying model, gpt-oss-120b, at 51,200 generations.
  • Gemini-3.1-Pro
    Average runtime across six datasets fell from 3,587.1 to 2,931.0 milliseconds, and discovery-agent calls dropped from 550 to 317.
  • Gemini-3.7-Flash
    Average runtime fell from 2,516.7 to 2,350.6 milliseconds, with calls down from 3,200 to 1,879.
  • GPU kernel tasks
    On VGG16 and LayerNorm, comparable performance with 2.43x and 1.79x fewer generations; on ConvDiv and ConvMax, 2.09x and 1.44x higher performance at comparable budgets.

Why it matters

Discovery loops are where agent budgets go to die. Teams refining a search can burn tens of thousands of tokens re-running lines of exploration that failed days ago. If replay genuinely substitutes for re-running, the logs sitting in every agent pipeline stop being an archive and start being an asset, and the cost curve for discovery-heavy work bends downwards.

There is a louder reason this lands now. Dario Amodei's essay We Must Pace the Frontier, published on 12 September, named recursive self-improvement as one of two developments that changed his mind about the need to slow AI, because it could produce systems too powerful for people to control. Dream-RSI is a narrower, more pragmatic slice of that idea: an agent improving its own exploration policy from records it already owns. The gap between existential runaway and disciplined bookkeeping is exactly where the RSI argument now lives.

Our read

The dreaming branding is doing marketing work, but the mechanism underneath is genuinely neat: the tree the agent already built is an exact simulator of the search space it reached, so improving the policy becomes a search through your own logs. Treat the headline figure with proportion. The 162x comparison runs against a baseline using a different model, while the like-for-like results show roughly 1.7x fewer calls on both Gemini variants, and that honest core is still a solid saving.

Replay also has a boundary the researchers name themselves: it can only answer counterfactuals inside the tree the agent has actually built, and every winning policy still has to go back online to prove itself. If you run discovery or agent-loop workloads, your own logs are the test bed. Run the replay on your history before you believe any multiplier, whatever its size.

What to watch

  • Independent replications
    Whether teams outside the author group reproduce the call reductions on their own workloads.
  • Framework adoption
    Whether replay-based policy improvement turns up as a feature in mainstream agent frameworks.
  • The safety engagement
    Whether researchers warning about RSI treat this as a rebuttal, a supporting exhibit or neither.
  • Scaling limits
    Where the approach breaks: stale trees and workloads where past outcomes stop predicting useful policies.

Discussion spark: Amodei named recursive self-improvement as a reason to slow the frontier, and this research makes a slice of it cheaper. Does efficiency work like this defuse the RSI worry or accelerate exactly what he warned about?

Sources and evidence

not affiliated with, endorsed by, or operated by Google or Google DeepMind

Google DeepMind Watch
#2817

Update

What changed

The fine print on Dream-RSI has arrived, and parts of it matter. Per David Borish's analysis on LinkedIn, the work is a preprint posted to arXiv on 14 September 2026 and has not been through peer review. His reading of the accompanying GitHub repository finds the paper and a project page live, with the full codebase, reproduction scripts and the discovered programs themselves still listed as in progress. That trims the earlier shorthand that the code is public: for now, much of what a curious reader could actually run is not there yet.

His account adds a second caveat that shapes how the headline numbers should be weighed. The comparisons against SimpleTES, AlphaEvolve and the other domain baselines lean on results reported in those systems' own papers, rather than figures the Dream-RSI team reproduced independently on matching hardware.

One experiment deserves a headline of its own. The researchers injected explicit guidance, high-level hints about which directions looked promising, straight into the discovery agent's prompt, and applied the same treatment to Dream-RSI and a fixed-strategy baseline. Both performed worse with the hints. The interpretation is that semantic steering layered onto a process already running many parallel search threads narrows exploration instead of sharpening it. Learning to read recorded history beat being told in words what the history implies.

Sources and evidence
  • The Replay Trick: Google DeepMind's Path to Cheaper Recursive Self-Improvement – LinkedIn: Google and Google DeepMind researchers' Dream-RSI is an unreviewed arXiv preprint posted 14 September 2026; per David Borish's 17 September analysis its repository still lists the full codebase, reproduction scripts and discovered programs as in progress, and the paper reports that explicit prompt hints worsened both Dream-RSI and a fixed-strategy baseline while Dream-RSI matched or edged out AlphaEvolve and AlphaEvolveV2 on mathematical optimisation tasks using under 1,000 generations.

Independent WittyWires Watcher; not an official account or feed.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.