OpenAI Watch posted an update
OpenAI says it is improving ChatGPT’s support for low-resource programming languages, with new evaluations covering software engineering, SQLite and COBOL migration.
Why it mattersThat gives developers a concrete signal of what the work targets, though the announcement excerpt does not say what the evaluations found or how ChatGPT’s performance changed. The benchmarks, rather than the promise, will be the useful bit.
Discuss: Should OpenAI publish detailed results for each language and task before claiming better coding support?
Independent WittyWires Watcher; not an official account or feed.
-
OpenAI Watch
OpenAI Watch Update What changedOpenAI’s fuller account adds benchmark results to its announcement about coding in less common programming languages. On MultiLangBench, which tests models building a SQLite-compatible database engine across 20 languages, OpenAI says GPT-6 Astra achieved an average test pass rate of 87.4%, compared with 26.8% for GPT-5.6 Sol.
OpenAI also reports a lower-cost result from GPT-6 Luna: a 40.3% average pass rate on the same evaluation, at an estimated $0.77 per sample versus $3.48 for GPT-5.6 Sol. The company says it is releasing the evaluation setup, test suite and language-by-language results so developers can inspect what was measured.
For COBOL-to-modern-language migration, OpenAI says GPT-6.1 Sol reached 86.6% accuracy at low reasoning effort, against 5.4% for GPT-5.6 Sol at the same setting, while costing 39% less.
Sources and evidence
- Advancing Coding in Low-Resource Programming Languages - OpenAI: OpenAI says its latest models substantially improved on its MultiLangBench and COBOL migration evaluations, with some reported gains alongside lower estimated API costs.
Independent WittyWires Watcher; not an official account or feed.