Perspective Toward Machine Learning Implementation in Pediatric Medicine: Mixed Methods Study

Natasha Alexander; Catherine Aftandilian; Lin Lawrence Guo; Erin Plenert; Jose Posada; Jason Fries; Scott Fleming; Alistair Johnson; Nigam Shah; Lillian Sung

doi:10.2196/40039

Perspective Toward Machine Learning Implementation in Pediatric Medicine: Mixed Methods Study

JMIR Med Inform. 2022 Nov 17;10(11):e40039. doi: 10.2196/40039.

Authors

Affiliations

¹ Division of Haematology/Oncology, The Hospital for Sick Children, Toronto, ON, Canada.
² Division of Hematology/Oncology, Department of Pediatrics, Stanford University, Palo Alto, CA, United States.
³ Program in Child Health Evaluative Sciences, Peter Gilgan Centre for Research and Learning, Toronto, ON, Canada.
⁴ Department of Systems Engineering and Computing, Universidad del Norte, Barranquilla, Colombia.
⁵ Stanford Center for Biomedical Informatics Research, Department of Biomedical Data Science, Stanford University, Palo Alto, CA, United States.

PMID: 36394938
PMCID: PMC9716421
DOI: 10.2196/40039

Abstract

Background: Given the costs of machine learning implementation, a systematic approach to prioritizing which models to implement into clinical practice may be valuable.

Objective: The primary objective was to determine the health care attributes respondents at 2 pediatric institutions rate as important when prioritizing machine learning model implementation. The secondary objective was to describe their perspectives on implementation using a qualitative approach.

Methods: In this mixed methods study, we distributed a survey to health system leaders, physicians, and data scientists at 2 pediatric institutions. We asked respondents to rank the following 5 attributes in terms of implementation usefulness: the clinical problem was common, the clinical problem caused substantial morbidity and mortality, risk stratification led to different actions that could reasonably improve patient outcomes, reducing physician workload, and saving money. Important attributes were those ranked as first or second most important. Individual qualitative interviews were conducted with a subsample of respondents.

Results: Among 613 eligible respondents, 275 (44.9%) responded. Qualitative interviews were conducted with 17 respondents. The most common important attributes were risk stratification leading to different actions (205/275, 74.5%) and clinical problem causing substantial morbidity or mortality (177/275, 64.4%). The attributes considered least important were reducing physician workload and saving money. Qualitative interviews consistently prioritized implementations that improved patient outcomes.

Conclusions: Respondents prioritized machine learning model implementation where risk stratification would lead to different actions and clinical problems that caused substantial morbidity and mortality. Implementations that improved patient outcomes were prioritized. These results can help provide a framework for machine learning model implementation.

Keywords: clinical utilization; machine learning; preferences; qualitative interviews.

©Natasha Alexander, Catherine Aftandilian, Lin Lawrence Guo, Erin Plenert, Jose Posada, Jason Fries, Scott Fleming, Alistair Johnson, Nigam Shah, Lillian Sung. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 17.11.2022.