Baseten has launched joey-1, a 2-billion-parameter model designed to return a label and confidence score instead of generating a block of text. The company says it serves an average 2,000-token request in 20 milliseconds at the 50th percentile, and costs $0.01 per million tokens on Baseten Model APIs.
Baseten Watch analysis
What happened
Baseten says joey-1 is the smallest of three models in its decision-model family, alongside elephant-1 (9B) and mammoth-1 (27B). It is available now through Baseten Model APIs, with open weights on Hugging Face. The company says all three models accept text and images and share the same API, so switching between them is a one-line change.
The intended use is a task with a defined answer, such as classifying a support ticket, flagging a message or approving a request. Baseten’s example estimates that classifying 10 million support tickets a month, at about 500 tokens each, would cost roughly $50 on joey-1 versus $500 on mammoth-1. That is the company’s calculation, not a universal bill: actual costs depend on the workload and deployment.
Baseten reports 20ms median end-to-end latency for an average 2,000-token request, compared with 49ms for elephant-1. It also calls joey-1 its strongest decision model with three billion or fewer parameters, based on internal testing, and says more details on those tests are coming.
Why it matters
For a system making many small decisions in a chain, generating prose at every step can add time and cost without adding much value. A model that returns a classification and confidence score directly could make those workflows cheaper and quicker, especially when a larger model is available to handle uncertain cases.
The example’s appeal is its arithmetic: at the stated prices, the smaller model could make a high-volume classification task far less expensive. The trade-off is that a lower-cost decision is useful only if it is accurate enough for the job. Baseten has not yet shared the details of the internal testing behind its performance claim.
Our read
This is a practical pitch for matching the model to the task, not asking one large language model to write its way through every yes-or-no decision. The shared API and escalation option make the family easier to test than three unrelated models. Start with a representative workload, measure accuracy as well as latency and spend, and keep a route to a larger model for uncertain calls. The cost saving is attractive; it still has to earn its keep on your data.
What to watch
- Whether Baseten publishes the internal evaluation details behind its model-quality claims.
- How joey-1 performs on real workloads against the larger models and other small decision models.
- Whether the advertised latency and costs hold for specific deployments and request patterns.
Discussion spark: For high-volume classification, would you trust a small decision model with an escalation path, or keep a larger model in the loop for every call?
Sources and evidence
- Source update (29 October 2026, 17:24 UTC)
not affiliated with or endorsed by Baseten