Improvement of an interobserver agreement of ARDS diagnosis by adding additional imaging and a confidence scale

Laura A Hagens; Fleur L I M Van der Ven; Nanon F L Heijnen; Marry R Smit; Hester A Gietema; Suzanne C Gerretsen; Marcus J Schultz; Dennis C J J Bergmans; Ronny M Schnabel; Lieuwe D J Bos

doi:10.3389/fmed.2022.950827

Improvement of an interobserver agreement of ARDS diagnosis by adding additional imaging and a confidence scale

Front Med (Lausanne). 2022 Aug 31:9:950827. doi: 10.3389/fmed.2022.950827. eCollection 2022.

Authors

Laura A Hagens¹, Fleur L I M Van der Ven^{1

2}, Nanon F L Heijnen³, Marry R Smit¹, Hester A Gietema^{4

5}, Suzanne C Gerretsen⁴, Marcus J Schultz^{1

6

7

8}, Dennis C J J Bergmans^{3

9}, Ronny M Schnabel³, Lieuwe D J Bos^{1

10}

Affiliations

¹ Department of Intensive Care, Amsterdam University Medical Center, Location Amsterdam Medical Center, University of Amsterdam, Amsterdam, Netherlands.
² Department of Intensive Care, Rode Kruis Ziekenhuis, Brandwondencentrum, Beverwijk, Netherlands.
³ Department of Intensive Care, Maastricht University Medical Centre+, Maastricht, Netherlands.
⁴ Department of Radiology and Nuclear Medicine, Maastricht University Medical Centre+, Maastricht, Netherlands.
⁵ GROW School for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands.
⁶ Mahidol-Oxford Tropical Medicine Research Unit (MORU), Mahidol University, Bangkok, Thailand.
⁷ Nuffield Department of Medicine, University of Oxford, Oxford, United Kingdom.
⁸ Medical Affairs, Hamilton Medical AG, Bonaduz, Switzerland.
⁹ School of Nutrition and Translational Research in Metabolism (NUTRIM), Maastricht University, Maastricht, Netherlands.
¹⁰ Department of Respiratory Medicine, Amsterdam UMC, Location AMC, University of Amsterdam, Amsterdam, Netherlands.

Abstract

Acute respiratory distress syndrome (ARDS) often is not recognized in clinical practice, largely due to variation in the interpretation of chest x-ray (CXR) leading to poor interobserver reliability. We hypothesized that the agreement in the interpretation of chest imaging for the diagnosis of ARDS in invasively ventilated intensive care unit patients between experts improves when using an 8-grade confidence scale compared to using a dichotomous assessment and that the agreement increases after adding chest computed tomography (CT) or lung ultrasound (LUS) to CXR. Three experts scored ARDS according to the Berlin definition based on case records from an observational cohort study using a dichotomous assessment and an 8-grade confidence scale. The intraclass correlation (ICC), imaging modality, and the scoring method were calculated per day and compared using bootstrapping. A consensus judgement on the presence of ARDS was based on the combined confidence grades of the experts, followed by a consensus meeting for conflicting scores. In total, 401 patients were included in the analysis. The best ICC was found using an 8-grade confidence scale for LUS (ICC: 0.49; 95%-CI: 0.29-0.63) and CT evaluation (ICC: 0.49; 95%-CI: 0.34-0.61). The ICC of CXR increased by 0.022 and of CT by 0.065 when 8-grade scoring was used instead of the dichotomous assessment. Adding information from LUS or chest CT increased the ICC by 0.25 when using the 8-grade confidence assessment. An agreement on the diagnosis of ARDS can increase substantially by adapting the scoring system from a dichotomous assessment to an 8-grade confidence scale and by adding additional imaging modalities such as LUS or chest CT. This suggests that a simple assessment of the diagnosis of ARDS with a chart review by one assessor is insufficient to define ARDS in future studies.

Clinical trial registration: Trialregister.nl (identifier NL8226).

Keywords: ARDS; CT; chest X-ray; confidence; diagnosis; imaging.