Deep learning for the detection of benign and malignant pulmonary nodules in non-screening chest CT scans

Ward Hendrix; Nils Hendrix; Ernst T Scholten; Mariëlle Mourits; Joline Trap-de Jong; Steven Schalekamp; Mike Korst; Maarten van Leuken; Bram van Ginneken; Mathias Prokop; Matthieu Rutten; Colin Jacobs

doi:10.1038/s43856-023-00388-5

Deep learning for the detection of benign and malignant pulmonary nodules in non-screening chest CT scans

Commun Med (Lond). 2023 Oct 27;3(1):156. doi: 10.1038/s43856-023-00388-5.

Authors

Ward Hendrix^{1

2}, Nils Hendrix^{1

2

3}, Ernst T Scholten¹, Mariëlle Mourits⁴, Joline Trap-de Jong⁵, Steven Schalekamp¹, Mike Korst², Maarten van Leuken⁴, Bram van Ginneken¹, Mathias Prokop^{1

6}, Matthieu Rutten^{1

2}, Colin Jacobs⁷

Affiliations

¹ Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands.
² Radiology Department, Jeroen Bosch Hospital, Henri Dunantstraat 1, 5223 GZ, 's-Hertogenbosch, The Netherlands.
³ Jheronimus Academy of Data Science, Sint Janssingel 92, 5211 DA, 's-Hertogenbosch, The Netherlands.
⁴ Radiology Department, Canisius Wilhelmina Hospital, Weg door Jonkerbos 100, 6532 SZ, Nijmegen, The Netherlands.
⁵ Radiology Department, St. Antonius Hospital, Koekoekslaan 1, 3435 CM, Nieuwegein, The Netherlands.
⁶ Radiology Department, University Medical Center Groningen, Hanzeplein 1, 9713 GZ, Groningen, The Netherlands.
⁷ Diagnostic Imaging Analysis Group, Radiology and Nuclear Medicine Department, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA, Nijmegen, The Netherlands. colin.jacobs@radboudumc.nl.

Abstract

Background: Outside a screening program, early-stage lung cancer is generally diagnosed after the detection of incidental nodules in clinically ordered chest CT scans. Despite the advances in artificial intelligence (AI) systems for lung cancer detection, clinical validation of these systems is lacking in a non-screening setting.

Method: We developed a deep learning-based AI system and assessed its performance for the detection of actionable benign nodules (requiring follow-up), small lung cancers, and pulmonary metastases in CT scans acquired in two Dutch hospitals (internal and external validation). A panel of five thoracic radiologists labeled all nodules, and two additional radiologists verified the nodule malignancy status and searched for any missed cancers using data from the national Netherlands Cancer Registry. The detection performance was evaluated by measuring the sensitivity at predefined false positive rates on a free receiver operating characteristic curve and was compared with the panel of radiologists.

Results: On the external test set (100 scans from 100 patients), the sensitivity of the AI system for detecting benign nodules, primary lung cancers, and metastases is respectively 94.3% (82/87, 95% CI: 88.1-98.8%), 96.9% (31/32, 95% CI: 91.7-100%), and 92.0% (104/113, 95% CI: 88.5-95.5%) at a clinically acceptable operating point of 1 false positive per scan (FP/s). These sensitivities are comparable to or higher than the radiologists, albeit with a slightly higher FP/s (average difference of 0.6).

Conclusions: The AI system reliably detects benign and malignant pulmonary nodules in clinically indicated CT scans and can potentially assist radiologists in this setting.

Plain language summary

Early-stage lung cancer can be diagnosed after identifying an abnormal spot on a chest CT scan ordered for other medical reasons. These spots or lung nodules can be overlooked by radiologists, as they are not necessarily the focus of an examination and can be as small as a few millimeters. Software using Artificial Intelligence (AI) technology has proven to be successful for aiding radiologists in this task, but its performance is understudied outside a lung cancer screening setting. We therefore developed and validated AI software for the detection of cancerous nodules or non-cancerous nodules that would need attention. We show that the software can reliably detect these nodules in a non-screening setting and could potentially aid radiologists in daily clinical practice.