Microsoft’s MAI-DxO achieved higher accuracy than generalist physicians on simulated complex diagnoses, with prospective hospital evaluations planned next.
Microsoft says its AI diagnostic system, MAI-DxO, scored 83% on 304 complex cases drawn from New England Journal of Medicine proceedings, compared with 36% for generalist physicians who could consult textbooks and online references. The findings were published in Science on October 22. On a separate set of emergency-department presentations from the MIMIC-IV-ED dataset, Microsoft reported 80% accuracy for the system versus 47% for physicians.
MAI-DxO is designed to imitate a step-by-step diagnostic process rather than answer from a complete case summary. It coordinates multiple AI models acting like a virtual panel: proposing diagnoses, requesting information or tests, and revising their views as evidence arrives. Microsoft’s Sequential Diagnosis Benchmark measures diagnostic accuracy alongside the evidence requested and estimated testing costs. The company said a cost-focused configuration maintained 81% accuracy on the complex cases at less than half the cost of comparative systems.
The results are notable, but they do not establish that the system is ready to diagnose or treat patients autonomously. The comparison used simulated benchmark cases, including unusually difficult published cases; the physicians were individuals, not specialist teams. The study therefore does not show how MAI-DxO would perform across routine patient populations, where diagnoses, available information, and risks can differ. Microsoft says it will evaluate the system prospectively with Beth Israel Deaconess Medical Center, comparing its recommendations with physicians’ independent assessments while patients receive standard care.
For healthcare professionals and health-system leaders, the next test is whether this approach can improve decisions in real clinical workflows without adding unsafe recommendations, unnecessary testing, or delays. Microsoft says it is open-sourcing parts of the benchmark so researchers can reproduce and extend the evaluation. The findings add to a fast-moving debate over clinical AI: strong performance on diagnostic challenges may justify further study, but evidence from real-world care and patient outcomes remains essential before broader deployment.