Validation of a candidate instrument to assess image quality in digital mammography using ROC analysis

Joana Boita; Ruben E van Engen; Alistair Mackenzie; Anders Tingberg; Hilde Bosmans; Anetta Bolejko; Sophia Zackrisson; Matthew G Wallis; Debra M Ikeda; Chantal van Ongeval; Ruud Pijnappel; Mireille Broeders; Ioannis Sechopoulos

doi:10.1016/j.ejrad.2021.109686

Validation of a candidate instrument to assess image quality in digital mammography using ROC analysis

Eur J Radiol. 2021 Jun:139:109686. doi: 10.1016/j.ejrad.2021.109686. Epub 2021 Mar 30.

Authors

Affiliations

¹ Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands; Dutch Expert Centre for Screening (LRCB), Wijchenseweg 101, 6538 SW, Nijmegen, the Netherlands.
² Dutch Expert Centre for Screening (LRCB), Wijchenseweg 101, 6538 SW, Nijmegen, the Netherlands.
³ National Coordinating Centre for the Physics of Mammography, Royal Surrey NHS Foundation Trust, Guildford, GU2 7XX, UK.
⁴ Department of Medical Radiation Physics, Translational Medicine Malmö, Lund University, Skåne University Hospital, Carl Bertil Laurells gata 9, SE-20502 Malmö, Sweden.
⁵ Department of Imaging and Pathology, Radiology, KUL, Herestraat 49, Leuven B-3000, Belgium; Department of Radiology, Radiology, UZ Gasthuisberg, Herestraat 49, Leuven B-3000, Belgium.
⁶ Department of Medical Imaging and Physiology, Translational Medicine Malmö, Lund University, Skåne University Hospital, Carl Bertil Laurells gata 9, SE-20502 Malmö, Sweden.
⁷ Cambridge Breast Unit, Cambridge University Hospitals NHS Foundation Trust, Cambridge & NIHR Cambridge Biomedical Research Centre, Cambridge, CB2 0QQ, UK.
⁸ Department of Radiology, Stanford University School of Medicine, 875 Blake Wilbur Dr, Stanford, CA, 94305, USA.
⁹ Department of Radiology, Radiology, UZ Gasthuisberg, Herestraat 49, Leuven B-3000, Belgium.
¹⁰ Dutch Expert Centre for Screening (LRCB), Wijchenseweg 101, 6538 SW, Nijmegen, the Netherlands; Department of Radiology, University Medical Center Utrecht, PO Box 85500, 3508 GA, Utrecht, Utrecht University, the Netherlands.
¹¹ Dutch Expert Centre for Screening (LRCB), Wijchenseweg 101, 6538 SW, Nijmegen, the Netherlands; Department for Health Evidence, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands.
¹² Department of Medical Imaging, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands; Dutch Expert Centre for Screening (LRCB), Wijchenseweg 101, 6538 SW, Nijmegen, the Netherlands. Electronic address: ioannis.sechopoulos@radboudumc.nl.

PMID: 33819803
DOI: 10.1016/j.ejrad.2021.109686

Abstract

Purpose: To validate a candidate instrument, to be used by different professionals to assess image quality in digital mammography (DM), against detection performance results.

Methods: A receiver operating characteristics (ROC) study was conducted to assess the detection performance in DM images with four different image quality levels due to different quality issues. Fourteen expert breast radiologists from five countries assessed a set of 80 DM cases, containing 60 lesions (40 cancers, 20 benign findings) and 20 normal cases. A visual grading analysis (VGA) study using a previously-described candidate instrument was conducted to evaluate a subset of 25 of the images used in the ROC study. Eight radiologists that had participated in the ROC study, and seven expert breast-imaging physicists, evaluated this subset. The VGA score (VGAS) and the ROC and visual grading characteristics (VGC) areas under the curve (AUC_ROC and AUC_VGC) were compared.

Results: No large differences in image quality among the four levels were detected by either ROC or VGA studies. However, the ranking of the four levels was consistent: level 1 (partial AUC_ROC: 0.070, VGAS: 6.77) performed better than levels 2 (0.066, 6.15), 3 (0.061, 5.82), and 4 (0.062, 5.37). Similarity between radiologists' and physicists' assessments was found (average VGAS difference of 10 %).

Conclusions: The results from the candidate instrument were found to correlate with those from ROC analysis, when used by either observer group. Therefore, it may be used by different professionals, such as radiologists, radiographers, and physicists, to assess clinically-relevant image quality variations in DM.

Keywords: Digital mammography; Image quality evaluation; Receiver operating characteristic; Type testing; Validation study; Visual grading analysis.

MeSH terms

Breast / diagnostic imaging
Breast Neoplasms* / diagnostic imaging
Humans
Mammography*
ROC Curve
Radiographic Image Enhancement
Radiologists

Grants and funding

28396/CRUK_/Cancer Research UK/United Kingdom