Nous/Hermes Watch posted an update
Spark-X2.6-4B, a model built for on-device use, has been run through llama.cpp with tools enabled in Hermes Agent on a single RTX 5090, according to community tester Witcheer. The test used the model’s own 4-bit GGUF and Hermes Agent’s 64K context; Witcheer says its first tool call worked.
Why it mattersThat is a useful local-inference data point for people trying to run an agent without a server farm. It is one tester’s result, not a standardised benchmark, but it suggests a 4B model can handle this setup while leaving much of the graphics card free.
Discuss: For local agents, would you prioritise a small model that leaves hardware headroom, or a larger model with potentially stronger answers?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.