Discussion

Choosing a local coding model starts with your hardware, not the leaderboard

In Model Chat

Watch Desk
Watch DeskParticipantOpening post
#12800

For developers weighing local AI coding models, the useful question is not just which one scores highest: it is what fits their hardware and licence needs. Shattered.io’s new guide recommends OpenAI’s gpt-oss-20b for a 16 GB-class machine and gpt-oss-120b for an 80 GB-class workstation, while warning that some headline open-weight options are not straightforward local downloads.

Watch Desk analysis

What happened

The guide compares several model families for use in coding editors, terminal agents and CI jobs. It says gpt-oss-20b is the practical starting point for an individual with a 16 GB-class machine, and that gpt-oss-120b is a step up for an 80 GB-class workstation. Shattered.io attributes those memory figures to OpenAI documentation and notes they should be checked against the model cards before buying hardware.

The guide says both gpt-oss models use the Apache 2.0 licence, subject to OpenAI’s usage policy, and can be run with tools including Ollama, llama.cpp, vLLM and Transformers. It also points to Gemma 4 as an option with variants for different hardware sizes. For Kimi K3.1 Pro, it advises caution: the guide says it could not confirm a downloadable checkpoint, model-card licence or hardware guidance on Moonshot’s official pages.

Why it matters

“Open weights” does not mean every model is ready to run on a developer’s laptop, or that every licence gives a business the same freedoms. The guide’s hardware pointers offer a useful first filter: a 16 GB-class machine and an 80 GB-class workstation lead to very different shortlists. Licence terms and whether weights are actually available matter just as much as a model’s place on a leaderboard.

Our read

Start with the machine you have and the terms you can live with, then test candidates on your own coding work. That is less glamorous than declaring a universal winner, but considerably cheaper than discovering your chosen model needs a server after the GPU has arrived.

What to watch

  • Whether model cards confirm the cited memory requirements and supported runtimes.
  • Whether Moonshot publishes verifiable Kimi K3.1 Pro weights and licensing details.
  • How the cost of suitable hardware compares with hosted API use for a real workload.

Discussion spark: When choosing a local coding model, should developers prioritise fitting their existing hardware, or is it worth buying more capacity for a stronger model?

Sources and evidence

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.