Self-hosting a frontier language model is not simply a matter of downloading weights and finding a spare graphics card. A new comparison says the largest models require multi-accelerator server setups, while prices, licensing and evidence behind performance claims vary sharply between providers.
Watch Desk analysis
What happened
Shattered.io compares Kimi K3.1 Pro, GLM-5.3, DeepSeek V4.1 Flash and Mistral Large 4 as options for teams weighing hosted APIs against running models on their own hardware. The guide says the smallest weights-only estimate among the models it discusses is 160 GB at 4-bit precision for GLM-5.3-Flash, before adding memory for the KV cache and serving overhead. That puts these models beyond a single consumer GPU, it argues.
The comparison also flags gaps in model documentation, differing licence terms and vendor-published benchmark results that were not all measured under one independent testing setup. Its figures and assessments are the guide’s account, not a neutral head-to-head benchmark.
Why it matters
The useful question is not just which model tops a leaderboard. Teams need to know whether they can run it, what they may legally do with it, and whether the expected workload makes self-hosting cheaper than paying per token. The article’s 160 GB minimum estimate is a practical reality check before anyone starts shopping for a home server with frontier ambitions.
Our read
The guide makes a persuasive case for checking hardware, licence terms and your own traffic before treating “open weights” as shorthand for easy or inexpensive deployment. Its figures are a starting point, not a procurement plan: the guide itself notes that cache, serving overhead and missing vendor details can change the calculation. Your own workload remains a better benchmark than a league table with a different test for every entrant.
What to watch
- Whether vendors publish clearer parameter counts, memory requirements and licence terms.
- How hosted API prices compare with the full cost of running a model for a real workload.
- Whether independent tests put the four model families through the same benchmarks. Self-hosting promises control, but the hardware and licensing details decide whether that control is worth buying. Is the freedom to run open-weight models worth the infrastructure and due diligence, or will hosted APIs remain the sensible choice for most teams?
Discussion spark: Is the freedom to run open-weight models worth the infrastructure and licensing work, or will hosted APIs remain the sensible choice for most teams?
Sources and evidence
- Self-Hosted LLM vs API: $0.70 to $2.00 Input [2026] – shattered.io (10 November 2026, 05:13 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.