Small open models take on reverse-engineering tasks, with mixed results

Hugging Face researchers tested whether small local models can help reverse-engineer compiled software, and found that a team of small agents can tackle a larger task, though not necessarily faster or cheaper. The experiments offer a useful reality check for anyone building AI coding teams: coordination and tooling matter, but they do not make the hard bits disappear.

Hugging Face Watch analysis

What happened

In a Hugging Face community article, Younes B, Nathan Habib and Lysandre describe two tests. A lead agent using DeepSeek-V4.1-Flash divided work among four Qwen3.5-9B workers, which tried to reproduce functions from compiled toy programs. An automated test tool compared each rewrite against the original, including the resulting game state.

For the 38-function animated-logo task, the team finished in 86 minutes at a reported cost of $2.97. A solo DeepSeek run took 14 minutes and cost $0.87. On the larger, 56-function space-shooter task, the team finished in 100 minutes for $3.60; the solo run took 54 minutes and cost $3.28. The lead sometimes improved the workers’ instructions and tools after spotting recurring mistakes, including mishandling signed versus unsigned comparisons.

Key findings

• Small agents needed oversight
  Workers escalated 15 of 38 functions in the first task and 20 of 56 in the second to the lead.
• The larger task narrowed the cost gap
  The team and solo run cost $3.60 and $3.28 respectively, but the team took almost twice as long.
• Better instructions helped with repeated errors
  The lead changed prompts and added a lookup tool to address recurring mistakes in offsets, signatures and comparison logic.

Why it matters

The results suggest that splitting a coding task can help when it grows beyond what a solo run handles comfortably. But the headline comparison is not a clean contest: these are two designed examples, and the second solo run used four launches while the team used two. The figures show what happened in these tests, not a general rule about agent teams or model costs.

The more transferable point may be less glamorous than “just add agents”: give workers reliable tools, shared context and a way to catch exact errors. Otherwise, a team can spend its time making the same wrong assumption in parallel. A cheerful little productivity multiplier this is not, at least not yet.

Our read

This is a valuable, concrete experiment rather than proof that small models can replace a capable lead. The larger task makes the team approach look more competitive on cost, while the slower completion and escalation rate show the price of coordination. Builders should treat these results as a prompt to test their own workflows, not as a spreadsheet-ready forecast.

What to watch

• Whether the authors test more complex programs and tasks beyond these two toy examples.
• Whether published runs make the cost, launch and time comparisons easier to reproduce.
• Whether better harnesses reduce escalations without adding enough overhead to erase the gains.

Discussion spark: For coding agents, would you choose a slower team of smaller models if it made the work easier to divide and check, or keep the task with one stronger model?

not affiliated with or endorsed by Hugging Face