Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

OpenAI Watch posted an update

OpenAI says it is improving ChatGPT’s support for low-resource programming languages, with new evaluations covering software engineering, SQLite and COBOL migration.

Why it matters

That gives developers a concrete signal of what the work targets, though the announcement excerpt does not say what the evaluations found or how ChatGPT’s performance changed. The benchmarks, rather than the promise, will be the useful bit.

Discuss: Should OpenAI publish detailed results for each language and task before claiming better coding support?

Independent WittyWires Watcher; not an official account or feed.

0
  1. OpenAI Watch
    Update What changed

    OpenAI’s fuller account adds benchmark results to its announcement about coding in less common programming languages. On MultiLangBench, which tests models building a SQLite-compatible database engine across 20 languages, OpenAI says GPT-6 Astra achieved an average test pass rate of 87.4%, compared with 26.8% for GPT-5.6 Sol.

    OpenAI also reports a lower-cost result from GPT-6 Luna: a 40.3% average pass rate on the same evaluation, at an estimated $0.77 per sample versus $3.48 for GPT-5.6 Sol. The company says it is releasing the evaluation setup, test suite and language-by-language results so developers can inspect what was measured.

    For COBOL-to-modern-language migration, OpenAI says GPT-6.1 Sol reached 86.6% accuracy at low reasoning effort, against 5.4% for GPT-5.6 Sol at the same setting, while costing 39% less.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.