Microsoft's AI tool can diagnose diseases with 80% accuracy, compared to doctors' 20% accuracy.
Microsoft has announced details of its artificial intelligence system , Microsoft AI Diagnostic Orchestrator (MAI-DxO) , which aims to create 'medical superintelligence' that outperforms human will in complex medical diagnoses. According to Microsoft, MAI-DxO can provide a diagnosis that is much cheaper than a doctor's diagnosis and is more than four times more accurate.
The Path to Medical Superintelligence | Microsoft AI
Microsoft said, 'As demand for healthcare grows, healthcare costs rise at an unsustainable pace, and billions of people face various barriers to improving their health, including inaccurate and delayed diagnoses. More people are turning to digital tools for medical advice and support. We want to help more.' In fact, Microsoft's consumer AI, such as Bing and Copilot, has more than 50 million health-related sessions every day.
More and more people are consulting AI chatbots about medical issues, but experts warn there are risks - GIGAZINE
Starting at the end of 2024, Microsoft's AI division launched a project led by clinicians, designers, engineers, and AI scientists as a consumer health-focused initiative. The first thing Microsoft focused on was establishing a benchmark for medical AI. Traditionally, the medical licensing exam required to practice as a doctor in the United States was often used as a benchmark, but the medical licensing exam is a multiple-choice memorization question that most AIs were able to answer with full marks. Going beyond that framework, Microsoft is working to evolve and evaluate clinical reasoning capabilities in practice, not in tests.
The Sequential Diagnostic Benchmark (SDBench) created by Microsoft takes cases from the New England Journal of Medicine (NEJM) , one of the most prestigious peer-reviewed medical journals, and converts them into a step-by-step diagnostic process. The model repeatedly asks questions under reasoning, updating the inference as new information is obtained, narrowing it down to a final diagnosis.
Introducing SDBench from Microsoft AI - YouTube
In addition to the baseline benchmark, Microsoft has developed a system called 'Microsoft AI Diagnostic Orchestrator (MAI-DxO)' that emulates a virtual panel of doctors with diverse diagnostic approaches working together to solve diagnostic cases. This converts any language model into a virtual panel of clinicians. Instead of individual models managing clinical workflows, multiple language models are combined to efficiently integrate diverse data sources, and it is said that it can be adapted to ever-changing medical needs while increasing safety and transparency.
Microsoft reports that by combining MAI-DxO with multiple language models, diagnostic performance improved significantly in all models tested. Particularly outstanding was the model that combined OpenAI's AI inference model o3 with MAI-DxO, correctly diagnosing 85.5% of NEJM benchmark cases. When 21 doctors with 5 to 20 years of clinical experience in the United States and the United Kingdom performed the same diagnostic task, they were able to correctly diagnose an average of 20% of cases, so Microsoft says that 'MAI-DxO achieved more than four times the accuracy of a human doctor.'
In addition, a characteristic of medical AI is that, if there is no defined cost constraint, it often performs any test regardless of the cost of treatment, the patient's wishes, the time it takes to treat, etc. Therefore, specifying the cost makes it easier to receive an appropriate diagnosis, but as shown in the graph below, MAI-DxO also has a better diagnostic performance per cost than other AI models and human diagnosis, which is indicated by the red cross.
While MAI-DxO excels at dealing with complex diagnostic challenges, Microsoft says further validation is needed to evaluate its performance in common and everyday cases. In response to the question of whether AI, which is cheaper and four times more powerful, will replace doctors, Microsoft said, 'The role of clinicians is much broader than just diagnosis, as they need to overcome ambiguous situations that AI cannot handle and build trusting relationships with patients and their families. AI is becoming a powerful tool in medicine, but our team of active clinicians sees AI as a complement to doctors and other medical professionals.'
'For us, this is just the first step. Significant challenges remain to safely and responsibly deploy generative AI across the healthcare sector, including the need for appropriate governance and regulatory frameworks, in addition to evidence from real clinical environments to ensure reliability, safety, and effectiveness. Together with our partners, we strongly believe that the future of healthcare will be shaped by augmenting human expertise and empathy with the power of machine intelligence. We are excited to take another step toward realizing this vision,' Microsoft said.
Related Posts:
