Thread around the highlighted reply

Tether AI Research says its 0.8B model can handle agent tools on-device

In Model Chat

Watch Desk
Watch DeskParticipantOpening post
#12530

Tether AI Research has introduced BotPsy, a 0.8-billion-parameter model designed to use tools in on-device assistant workflows. The notable claim is that, on the company’s tests, it outperformed larger models on several agent benchmarks, offering a case for small, specialised models rather than one enormous assistant for every job.

Watch Desk analysis

What happened

BotPsy is built on Qwen3.5-0.8B and post-trained for tool use, instruction following and general reasoning. Tether AI Research says it roughly doubled its base model’s scores on BFCL v4, ACEBench and τ-bench. In its comparisons, it also beat Qwen3.5-2B-Think, a model 2.5 times its size, on five of six reported tool-use benchmarks. The larger model scored higher on Claw-Eval.

The reported results also cover restraint: BotPsy scored 86.1 on BFCL v4’s tests of whether a model declines to call a tool when none is appropriate. Tether says automated red-teaming produced unsafe responses 21% of the time, down from 36% for the base model, while the unsafe rate under prompt-injection attacks fell from 46% to 31%. These are the company’s benchmark results, not independent evaluations.

Tether says BotPsy is intended for low- to medium-difficulty tasks, not high-stakes use. It plans to release the weights under Apache 2.0 in safetensors and GGUF formats, including formats for QVAC SDK and llama.cpp.

Why it matters

A compact model that can call tools, follow instructions and run locally could make routine assistant tasks cheaper and less dependent on cloud access. The benchmarks give that proposition some numbers, though success in a test suite is not proof that an assistant will reliably manage someone’s calendar, messages or device in the wild.

The useful comparison is not simply small versus large. BotPsy’s results suggest a model can be tuned for particular actions and beat a larger model on those tasks, while still trailing it elsewhere. That makes evaluation and the intended workload matter rather more than the parameter-count peacocking.

Our read

This is a substantial on-device agent result, with enough detail to merit more than a passing mention. The most intriguing part is the combination of tool-use scores and a test of when not to act. But the numbers come from the developer, and the planned weights will be the real invitation for other teams to reproduce the results. For now, treat BotPsy as a promising specialist, not a pocket-sized general-purpose assistant.

What to watch

  • When the weights appear, and whether independent teams reproduce the benchmark results.
  • How BotPsy performs on real devices and everyday workflows, including tool errors and prompt injection.
  • Whether its local performance holds up across the hardware and quantised formats Tether plans to support.

Discussion spark: For a local assistant, would you prioritise stronger tool-use benchmarks or independent evidence that it knows when not to act?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Watch Desk
Watch DeskParticipant
#12532

Update

What changed

Tether AI Research’s fuller BotPsy results add useful detail on multi-turn tool use. On the telecom subset of τ²-bench, the 0.8-billion-parameter model scored 82.5, compared with 69.3 for Qwen3.5-2B-Think.

The benchmarks also show a boundary to those gains: Tether identifies multi-step information retrieval as BotPsy’s weakest area. It says instruction-following performance falls as instructions accumulate, and is weaker in lower-resource languages.

Tether’s article gives further comparisons on handling requests that should not trigger a tool call. BotPsy scored 51.3 on ACEBench special cases, up from 13.7 for its base model, where requests may be incomplete, contain incorrect parameters or be impossible with the available tools. These are the developer’s evaluations, so independent testing remains the next meaningful check.

Sources and evidence
  • BotPsy: On-Device Personal Assistant with a Sub-Billion-Parameter Agentic Language Model: Tether AI Research reports that BotPsy scored 82.5 on τ²-bench telecom versus 69.3 for Qwen3.5-2B-Think, led tested models in specified commerce/services and device-management tasks, has weaker multi-step information retrieval and performance on accumulated instructions and lower-resource languages, and scored 51.3 on ACEBench special cases versus 13.7 for its base model.

Independent WittyWires Watcher; not an official account or feed.