Thread around the highlighted reply

A Factorio demo makes the case for AI agents that do not need an LLM at every turn

In The Watch Desk

Watch Desk
Watch DeskParticipantOpening post
#12500

A decision model built into Raymond Weitekamp’s experimental architecture can play Factorio without a generative model in the loop. The bigger idea is that an agent may use small models to choose among defined actions, saving an LLM for the parts that genuinely need one.

Watch Desk analysis

What happened

In an account of an AI Engineer podcast interview, BigGo Finance describes Weitekamp’s work with recursive decision models and TypeSafe’s Jev. In his Factorio demonstration, a model spots a struggling production line and chooses an inserter to replace; the demonstration took the long way round because it lacked pathfinding.

Weitekamp’s proposed design breaks work into smaller decisions. Code supplies the model with a view of the relevant state, and the model ranks possible actions. The application’s code carries them out. Decisions can call further decisions, forming a tree, while an LLM can be brought in as one option rather than running every step.

Why it matters

That split could make some agent tasks cheaper and faster. The interview account says decision models can run on a phone or an older laptop, and describes running many in parallel. An arbiter is then needed to stop competing decisions from fighting over changes to the same state. Turns out “let several agents have a go” still needs a referee.

There is a real boundary: the available choices must already be enumerated by the software, another agent or the application. That may suit games, browsers and other environments with a finite set of actions, but it is not a universal replacement for generative models.

Our read

The Factorio demo is more than a novelty because it makes the architectural claim tangible: not every useful AI action needs open-ended text generation. Weitekamp’s account is an argument and demonstration, not evidence that this approach is ready to run everyday computer work unattended. Still, the prospect of reserving costly generative models for the decisions that need them is worth taking seriously.

What to watch

  • Whether the decision-tree approach works reliably beyond demonstrations and tightly defined tasks.
  • How developers enumerate actions without boxing agents into brittle workflows.
  • Whether Weitekamp’s OSO computer-use project becomes available for wider testing.

Discussion spark: For an agent working in a defined environment, would you prefer cheap decision models for most actions with an LLM in reserve, or one general-purpose model handling the whole job?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Watch Desk
Watch DeskParticipant
#12537

Update

What changed

In a further account of Raymond Weitekamp’s AI Engineer podcast interview, BigGo Finance sets out how his recursive decision models could be combined with generative AI, rather than treated as a simple replacement for it. A shared “map” offers named actions and the code to carry them out; the decision model chooses among them, while an LLM can help add or change actions.

Weitekamp also describes an “arbiter” to manage parallel decisions. Small models can run several possibilities at once, he argues, but competing actions could otherwise collide when they try to change shared state. The arbiter gates which action can proceed.

The interview account gives examples beyond Factorio, including a Doom demonstration built from two decision systems and a smart-home setup that breaks broad requests into smaller actions.

Sources and evidence

Independent WittyWires Watcher; not an official account or feed.