An experimental machine learning study investigating the decision-making process of students and qualified radiographers when interpreting radiographic images

Clare Rainey; Angelina T Villikudathil; Jonathan McConnell; Ciara Hughes; Raymond Bond; Sonyia McFadden

doi:10.1371/journal.pdig.0000229

An experimental machine learning study investigating the decision-making process of students and qualified radiographers when interpreting radiographic images

PLOS Digit Health. 2023 Oct 25;2(10):e0000229. doi: 10.1371/journal.pdig.0000229. eCollection 2023 Oct.

Authors

Clare Rainey¹, Angelina T Villikudathil¹, Jonathan McConnell², Ciara Hughes¹, Raymond Bond³, Sonyia McFadden¹

Affiliations

¹ Faculty of Life and Health Sciences, School of Health Sciences, Ulster University, York Street, Belfast, Northern Ireland, United Kingdom.
² NHS Leeds Teaching Hospitals, Leeds, United Kingdom.
³ Faculty of Computing, School of Computing, Engineering and the Built Environment, Ulster University, York Street, Belfast, Northern Ireland, United Kingdom.

Abstract

AI is becoming more prevalent in healthcare and is predicted to be further integrated into workflows to ease the pressure on an already stretched service. The National Health Service in the UK has prioritised AI and Digital health as part of its Long-Term Plan. Few studies have examined the human interaction with such systems in healthcare, despite reports of biases being present with the use of AI in other technologically advanced fields, such as finance and aviation. Understanding is needed of how certain user characteristics may impact how radiographers engage with AI systems in use in the clinical setting to mitigate against problems before they arise. The aim of this study is to determine correlations of skills, confidence in AI and perceived knowledge amongst student and qualified radiographers in the UK healthcare system. A machine learning based AI model was built to predict if the interpreter was either a student (n = 67) or a qualified radiographer (n = 39) in advance, using important variables from a feature selection technique named Boruta. A survey, which required the participant to interpret a series of plain radiographic examinations with and without AI assistance, was created on the Qualtrics survey platform and promoted via social media (Twitter/LinkedIn), therefore adopting convenience, snowball sampling This survey was open to all UK radiographers, including students and retired radiographers. Pearson's correlation analysis revealed that males who were proficient in their profession were more likely than females to trust AI. Trust in AI was negatively correlated with age and with level of experience. A machine learning model was built, the best model predicted the image interpreter to be qualified radiographers with 0.93 area under curve and a prediction accuracy of 93%. Further testing in prospective validation cohorts using a larger sample size is required to determine the clinical utility of the proposed machine learning model.

Copyright: © 2023 Rainey et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Grants and funding

This work was supported by the College of Radiographers Research Industry Partnership award Scheme (CoRIPS: Grant number 213) (CR (P.I.), SMF, CH, RB, JMcC). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.