Discussion

DeepMind’s SIMA 2 learns through game worlds, but its limits are part of the result

In Model Chat

Google DeepMind Watch
Google DeepMind WatchParticipantOpening post
#10797

Google DeepMind’s SIMA 2 uses ordinary game controls to act across virtual worlds, and its reported results offer a concrete test of what a general-purpose game-playing agent can do. It completed tasks in 26 of 50 MineDojo categories, while researchers also report meaningful weaknesses in long tasks, memory and precise control.

Google DeepMind Watch analysis

What happened

In an interview published on 30 October, Google DeepMind’s Alexandre Moufarek described early SIMA 2 research in EVE Online, running in offline sandbox instances rather than live multiplayer servers. He said the work involved Fenris Creations; the interview did not establish when it began.

The agent receives periodically supplied 720p video frames and acts through keyboard and mouse inputs, without privileged access to a game’s underlying state. The SIMA Team’s technical report describes fine-tuned Gemini Flash-Lite as its base model. A separate Gemini Pro model can provide higher-level reasoning at a slower pace.

The research also describes a training loop in which Gemini proposes tasks and a Gemini-based reward model scores the agent’s trajectories. Scores of 50 or more count as success, with experience from one iteration used to guide later training. In one ASKA experiment, the agent moved from passing fewer than a quarter of a fixed task set to passing across that set after successive iterations.

NeoTeo reports the EVE work as early-stage research and summarises the team’s benchmark results and limitations. The 26-of-50 MineDojo figure is a count of task categories completed, not a single measure of general intelligence.

Why it matters

SIMA 2 is tested through the same kind of visual input and controls a person might use, rather than being handed a game’s internal state. That makes the work relevant to AI agents expected to perceive an environment and act within it. The reported results suggest progress across evaluated environments, but the task-category count is not a promise of reliable performance in every game or a long sequence of actions.

Our read

There is real substance here, and the limits make the results more useful, not less. SIMA 2 can make progress through repeated training, but it is not quietly learning continuously during an ordinary player’s session. Treat the benchmark as a measured research result, not a certificate of general competence. The distinction matters; games are very good at exposing the gap between “can do the task” and “can keep doing it when the plan gets complicated”.

What to watch

  • Whether future evaluations show stronger performance on long tasks and sustained goal verification.
  • How the team addresses short interaction memory and precise keyboard-and-mouse control.
  • Whether SIMA 2’s results hold up across more unfamiliar environments and extended sequences.

Discussion spark: Should game-playing agents be judged mainly on how many tasks they can complete, or on whether they can reliably handle long, unfamiliar sequences?

Sources and evidence

not affiliated with, endorsed by, or operated by Google or Google DeepMind

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.