New AI system beats doctors on hard medical cases

Patients facing complicated medical mysteries may soon get a second opinion from an artificial intelligence that outperforms human specialists in simulated environments. A new study published in Science this week reveals that a Microsoft diagnostic system correctly identified complex cases more than twice as often as generalist physicians. The results suggest a potential turning point in how difficult diagnoses are handled, moving the technology from theoretical promise toward practical application for everyday healthcare delivery.

The data behind these claims is stark. In a simulation using complex cases from the New England Journal of Medicine, the system, known as MAI-DxO, achieved an accuracy rate of 83%. By comparison, primary care doctors and general internists working individually with reference materials were correct in 36% of those same cases. It is important to note that such complex scenarios are typically handled by specialists or teams of physicians, who were not included in this specific comparison.

The system functions by coordinating multiple AI agents that operate like a panel of medical experts. These agents question the patient, order necessary tests, and debate the findings to reach a consensus. Dominic King, vice president of Health at Microsoft AI, described the findings as substantial and exciting. He highlighted that the core challenge in healthcare today is the growing gap between the demand for medical expertise and the available supply.

Microsoft is not alone in this space. The company joins Google and OpenAI in asserting that its AI can match or exceed human performance. However, the study indicates that much of the improvement stems from more powerful underlying AI models rather than the orchestration system alone. Microsoft tested its framework with models from several companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The best results came when using OpenAI’s GPT-5 and GPT-5.5 models.

Cost and efficiency were also part of the evaluation. On its own, GPT-5 scored 78.9% on complex cases at an estimated cost of $8,400 per case. The DxO system reduced that cost by about $3,000 while improving accuracy by only four points. For emergency department cases, there was little difference in accuracy or cost between the standalone model and the assisted system. With the newer GPT-5.5 model, the results were mixed. DxO raised accuracy from 72% to nearly 87% for complex cases at a slightly lower cost, but gains did not appear for emergency cases, where both versions scored around 85%.

The orchestration system proves most valuable when used with less capable models. For instance, Gemini 2.5 Flash scored 46.4% on its own but reached nearly 69% with DxO assistance. Researchers noted that accuracy gains naturally diminish as models approach theoretical maximums. A Microsoft spokesperson explained that the system helps optimize both accuracy and cost in the broader challenge of clinical decision making.

Mustafa Suleyman, CEO of Microsoft AI, views this as a step toward medical superintelligence. He is leading efforts to build a frontier AI foundation model for healthcare in partnership with the Mayo Clinic, with a release planned for next year. Suleyman believes this infrastructure will enable enormous leaps. He predicts that by 2027, there will be a breakthrough in coding for medical records, allowing for highly accurate predictions of how a patient's condition might unfold.

Suleyman, a former co-founder of DeepMind, has long focused on using AI to solve medical issues. He joined Microsoft in 2024, followed shortly by King. The two are true believers in the technology's potential. Suleyman expects healthcare models to improve by an order of magnitude in capability within the next 24 to 36 months. He cites the rapid progress seen in coding and chatbots as evidence that a flywheel of real-world usage and iteration will drive similar gains in medicine.

To validate these claims in real settings, Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston. The study will evaluate DxO across different hospital environments. Patients will receive standard treatment while consenting participants also engage with the AI. The company will compare the AI’s recommendations against human doctors. King called this the largest study of its kind, aiming to provide compelling evidence for deploying these systems at scale in emergency and urgent care settings.

This work aligns with a new manifesto from Microsoft defining medical superintelligence as the capacity to surpass human-led health systems in reasoning and coordination. The company aims to move beyond simple benchmarks to integrate with clinicians for better outcomes. Suleyman stated that this focus could differentiate Microsoft from other frontier labs. The company already sells dictation and radiology tools and sees DxO as a key expansion. Its goal is to have the number one state-of-the-art clinical decision support product in the coming years.