Inter-rater reliability and double reading analysis of an automated three-dimensional breast ultrasound system: comparison of two independent examiners

Arch Gynecol Obstet. 2017 Sep;296(3):571-582. doi: 10.1007/s00404-017-4473-y. Epub 2017 Jul 26.

Abstract

Purpose: Breast ultrasound could be a valuable tool complementary to mammography in breast cancer screening. Automated 3D breast ultrasound (ABUS) addresses challenges of hand-held ultrasound and could allow double reading analysis of ultrasound images. This trial assesses the inter-rater reliability and double reading analysis of an ABUS system.

Methods: To assess the reproducibility and diagnostic validity of the ABUS system, SomoV™, a blinded double reading analysis, was performed in 1019 patients (2038 breasts) by two examiners (examiner A/B) and compared to single reading results, as well as to the reference standard regarding its diagnostic validity. Cohen's kappa coefficients were calculated to measure the inter-rater reliability and agreement of the different diagnostic modalities. Patient comfort and time consumption for image acquisition and reading were analyzed descriptively as secondary objectives.

Results: Analysis of inter-rater reliability yielded agreement in 81.6% (κ = 0.37; p < 0.0001) showing fair agreement. Single reading analysis of SomoV™ exams (examiner A/examiner B) compared to reference standard showed good specificity (examiner A: 88.3%/examiner B: 84.5%), fair inter-rater agreement (examiner A: κ = 0.31/examiner B: κ = 0.31), and adequate sensitivity (examiner A: 53.1%/examiner B: 64.2%). Double reading analysis yielded good sensitivity and specificity (73.7 and 77.7%). Mammography (n = 1911) alone detected 160 of 176 carcinomas (sensitivity 90.1%). Adding SomoV™ to mammography would have detected 12 additional carcinomas, resulting in a higher sensitivity of 97.7%.

Conclusion: SomoV™ is a promising technique with good sensitivity, high patient comfort, and fair inter-examiner reliability. It allows double reading analysis that, in combination with mammography, could increase detection rates in breast cancer screening.

Keywords: Automated 3D breast ultrasound (ABUS); Breast cancer; Double reading analysis; Hand-held ultrasound; Inter-rater reliability.

MeSH terms

  • Breast / diagnostic imaging
  • Early Detection of Cancer
  • Female
  • Humans
  • Image Interpretation, Computer-Assisted
  • Imaging, Three-Dimensional* / methods
  • Imaging, Three-Dimensional* / standards
  • Reproducibility of Results
  • Sensitivity and Specificity
  • Ultrasonography, Mammary* / methods
  • Ultrasonography, Mammary* / standards