Evaluating the quality and readability of ChatGPT-generated patient-facing medical information in rhinology.

Evaluating the quality and readability of ChatGPT-generated patient-facing medical information in rhinology. European archives of oto-rhino-laryngology : official journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS) : affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery Fazilat, A. Z., Brenac, C., Kawamoto-Duran, D., Berry, C. E., Alyono, J., Chang, M. T., Liu, D. T., Patel, Z. M., Tringali, S., Wan, D. C., Fieux, M. 2024

Abstract

The artificial intelligence (AI) chatbot ChatGPT has become a major tool for generating responses in healthcare. This study assessed ChatGPT's ability to generate French preoperative patient-facing medical information (PFI) in rhinology at a comparable level to material provided by an academic source, the French Society of Otorhinolaryngology (Société Française d'Otorhinolaryngologie et Chirurgie Cervico-Faciale, SFORL).ChatGPT and SFORL French preoperative PFI in rhinology were compared by analyzing responses to 16 questions regarding common rhinology procedures: ethmoidectomy, sphenoidotomy, septoplasty, and endonasal dacryocystorhinostomy. Twenty rhinologists assessed the clarity, comprehensiveness, accuracy, and overall quality of the information, while 24 nonmedical individuals analyzed the clarity and overall quality. Six readability formulas were used to compare readability scores.Among rhinologists, no significant difference was found between ChatGPT and SFORL regarding clarity (7.61?±?0.36 vs. 7.53?±?0.28; p?=?0.485), comprehensiveness (7.32?±?0.77 vs. 7.58?±?0.50; p?=?0.872), and accuracy (inaccuracies: 60% vs. 40%; p?=?0.228), respectively. Non-medical individuals scored the clarity of ChatGPT significantly higher than that of the SFORL (8.16?±?1.16 vs. 6.32?±?1.33; p?

View details for DOI 10.1007/s00405-024-09180-0

View details for PubMedID 39724239

View details for PubMedCentralID 10315641