Memento 3’s researchers say their framework cleared every level across ARC-AGI-3’s 25 public games, while a Pong controller won 21-0 in evaluated episodes without further calls to a large language model. The intriguing idea is to let a frozen agent improve its behaviour through a revisable rulebook, rather than changing the underlying model.
Watch Desk analysis
What happened
In their arXiv paper, the researchers describe Memento 3 as a model-based framework in which an agent keeps an explicit world model in external memory. It maintains a natural-language rulebook, updating and compiling it into executable code using prediction errors and verification.
They report a mean Relative Human Action Efficiency of 100.0 on ARC-AGI-3, clearing all 25 public games with a single-model agent. Their Pong case study reports a learned feedback controller winning 21-0 in evaluated episodes without additional LLM calls.
Key findings
- A rulebook that can change
The framework updates external instructions from prediction errors, then compiles them into code to guide later actions. - Strong results on two tests
The paper reports clearing all 25 public ARC-AGI-3 games and a 21-0 Pong result in evaluated episodes.
Why it matters
The approach treats an agent’s accumulated experience as something it can turn into a more explicit, inspectable operating procedure. If that works beyond these tests, it could make systems more capable over repeated tasks without retraining the underlying language model or asking it to reason from scratch at every step. The Pong result is particularly notable for showing the learned controller acting without more LLM calls during the evaluated episodes.
These are reported results on named benchmarks and a case study, not evidence that the method generalises to arbitrary real-world tasks. A perfect score across the public games is striking; it also makes the boundary of the test, and performance on unfamiliar problems, especially important.
Our read
Memento 3 is an interesting bet on agent improvement by better memory and rules, rather than a bigger model. The results are worth attention, but the next question is whether the same method holds up when tasks are new, messy and less conveniently scored. Benchmarks are useful; the world, regrettably, does not come with an answer key.
What to watch
- Whether the researchers publish results on tasks beyond the public ARC-AGI-3 games.
- How reliably the rulebook updates and compiled code behave on unfamiliar problems.
- Whether the method’s gains persist across more tasks and independent evaluations.
Discussion spark: Should agent systems be judged partly on whether they can turn experience into reusable rules, or do benchmark results matter more than the mechanism behind them?
Sources and evidence
- Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks (9 October 2026, 02:28 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.