Microsoft Says AI Diagnostic Tool Outperforms Doctors in Science Study

The system, called Microsoft Diagnostic Orchestrator or MAI-DxO, uses multiple AI agents designed to work like a panel of medical experts. The agents reason through symptoms and medical histories, decide which tests to order and arrive at a diagnosis, mimicking the process doctors use. Microsoft compiled cases from medical reports and emergency department data into a benchmark called SDBench, which it plans to open-source, The Verge reported.

On cases drawn from hundreds of reports published in The New England Journal of Medicine, Microsoft’s system reached about 83% diagnostic accuracy, compared with roughly 36% for generalist physicians using references and 22% for those working without them. On emergency department cases from a patient database at Beth Israel Deaconess Medical Center in Boston, the system scored 80% accuracy, compared with an average of 47% for human physicians, according to the study. The AI system also tended to reach diagnoses at a lower estimated cost than human counterparts, The Verge reported.

The Verge reported that the system was powered by OpenAI’s GPT-5. TechCrunch reported that Microsoft tested models from several companies and that its best results came with OpenAI’s GPT-5 and GPT-5.5. TechCrunch also reported that much of the improvement came from more powerful underlying models, with the orchestration system adding gains on top, particularly on complex cases and with less capable models. Gains were smaller on emergency department cases. The Verge noted that the peer-reviewed paper does not include newer models such as GPT-6 Astra, Meta’s Muse Wonder Full, Google’s Gemini 4 Argon or Anthropic’s Opus 5.5.

TechCrunch reported a caveat in the comparison: the complex New England Journal of Medicine cases are usually handled by specialists or physician teams, who were not part of the comparison. Microsoft acknowledges that the system has been tested only on past medical cases and that it remains unclear how well it would perform in real clinical settings. “It needs clinical validation,” Mustafa Suleyman, Microsoft AI’s CEO, told The Verge. “That is a leap, but this is a major milestone along that journey.”

Microsoft plans to begin testing the system in hospital settings with Beth Israel Deaconess Medical Center to see how it performs in real-world clinical environments, Dominic King, vice president of Health at Microsoft AI and a coauthor of the study, told The Verge. King said the work would likely “get underway in the next couple of months,” but neither he nor Suleyman would commit to a firm timeline. Suleyman told TechCrunch that Microsoft is working with Mayo Clinic to train a foundation model for healthcare, with release planned for next year. Microsoft has not said when or how the diagnostic technology might become commercially available. King emphasized the need for a responsible rollout and noted the high regulatory and compliance requirements in medicine.