Notice: Users may be experiencing issues with displaying some pages on stanfordhealthcare.org. We are working closely with our technical teams to resolve the issue as quickly as possible. Thank you for your patience.
 

Learn about the flu shot, COVID-19 vaccine, and our masking policy »

Stanford Health Care

Human-machine partnership with artificial intelligence for chest radiograph diagnosis. NPJ digital medicine Patel, B. N., Rosenberg, L. n., Willcox, G. n., Baltaxe, D. n., Lyons, M. n., Irvin, J. n., Rajpurkar, P. n., Amrhein, T. n., Gupta, R. n., Halabi, S. n., Langlotz, C. n., Lo, E. n., Mammarappallil, J. n., Mariano, A. J., Riley, G. n., Seekins, J. n., Shen, L. n., Zucker, E. n., Lungren, M. n. 2019; 2: 111

Abstract

Human-in-the-loop (HITL) AI may enable an ideal symbiosis of human experts and AI models, harnessing the advantages of both while at the same time overcoming their respective limitations. The purpose of this study was to investigate a novel collective intelligence technology designed to amplify the diagnostic accuracy of networked human groups by forming real-time systems modeled on biological swarms. Using small groups of radiologists, the swarm-based technology was applied to the diagnosis of pneumonia on chest radiographs and compared against human experts alone, as well as two state-of-the-art deep learning AI models. Our work demonstrates that both the swarm-based technology and deep-learning technology achieved superior diagnostic accuracy than the human experts alone. Our work further demonstrates that when used in combination, the swarm-based technology and deep-learning technology outperformed either method alone. The superior diagnostic accuracy of the combined HITL AI solution compared to radiologists and AI alone has broad implications for the surging clinical AI deployment and implementation strategies in future practice.

View details for DOI 10.1038/s41746-019-0189-7

View details for PubMedID 31754637

View details for PubMedCentralID PMC6861262