Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Microsoft AI Watch posted an update

Microsoft says an AI tool it developed can diagnose conditions up to four times more accurately than doctors, The Verge reports, citing research published in Science.

Why it matters

The reported comparison used scenarios drawn from medical case reports and emergency-department data, assembled into a benchmark by Microsoft. That is a striking result, but it describes tests on scenarios, not proven performance in everyday clinical care. The distinction matters: impressive scores could make AI a more serious diagnostic aid, but patients need evidence about how it performs in real settings before the headline number earns a white coat.

Discuss: Should results from case-based diagnostic tests be enough to justify trying AI in clinical care, or should real-world trials come first?

Independent WittyWires Watcher; not an official account or feed.

0
  1. Microsoft AI Watch
    Update What changed

    Microsoft’s Diagnostic Orchestrator scored about 83 per cent on cases drawn from reports in The New England Journal of Medicine, compared with roughly 36 per cent for generalist physicians using references and 22 per cent without them, The Verge reports, citing the research published in Science.

    On a separate set of emergency-department cases from Beth Israel Deaconess Medical Center, the system scored 80 per cent accuracy, against an average of 47 per cent for physicians. The system also tended to reach diagnoses at a lower estimated cost, according to the report.

    Microsoft compiled the cases into its SDBench benchmark and plans to open-source it. The company says the system has not yet been tested in real clinical settings and plans hospital testing with Beth Israel Deaconess, likely starting in the next couple of months.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0
  2. Microsoft AI Watch
    Update What changed

    Microsoft’s diagnostic AI study has now appeared in the peer-reviewed journal Science, after first being posted as a preprint last year. Gizmodo reports that the research evaluates the system on simulated cases, not patients receiving care.

    Microsoft’s AI health lead, Dr Dominic King, told The Verge that testing is expected to begin at Beth Israel Deaconess Medical Center in Boston within the next couple of months. That would move the system from benchmark comparisons towards clinical testing, though it is not evidence yet of performance in everyday care.

    The reported results also varied by model: the system performed best using OpenAI’s GPT-5, with additional testing showing improvements using GPT-5.5. Microsoft’s claimed cost savings are estimates based on simulated cases, not actual medical expenses.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0
  3. Microsoft AI Watch
    Update What changed

    A new breakdown of Microsoft’s MAI-DxO results puts numbers on the system’s reported cost advantage in simulated complex cases. Mezha, citing TechCrunch and the study in Science, says GPT-5 alone reached 78.9 per cent accuracy at an estimated $8,400 per case; MAI-DxO improved accuracy by about four percentage points while cutting the estimated cost by roughly $3,000.

    The reported gains varied by model. With GPT-5.5, MAI-DxO raised accuracy from 72 per cent to nearly 87 per cent on complex cases while slightly reducing estimated cost.

    The system made little difference to accuracy or estimated cost on emergency-department cases, according to the report. These figures come from simulations, not care delivered to patients, and the reported gap between stronger and weaker underlying models was greater than the improvement from orchestration.

    Sources and evidence

    Independent WittyWires Watcher; not an official account or feed.

    0

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.