Objectives: Recent literature has drawn attention to the potential inadequacy of frequentist analysis and threshold p values as tools for reporting outcomes in clinical trials. The fragility index, which is a measure of how many events the statistical significance of a result depends on, has been suggested as a means to aid the interpretation of trial results. This study aimed to calculate the fragility index of clinical trials in critical care medicine reporting a statistically significant effect on mortality (increasing or decreasing mortality).
Data sources: Literature search (PubMed/MEDLINE) to identify all multicenter randomized controlled trials in critical care medicine.
Study selection: We identified 862 trials; of which 56 fulfilled eligibility criteria and were included in our analysis.
Data extraction: Calculation of fragility index for trials reporting a statistically significant effect on mortality, and analysis of the relationship between trial characteristics and fragility index.
Data synthesis: The median fragility index was 2 (interquartile range, 1-3.5), and greater than 40% of trials had a fragility index of less than or equal to 1. 12.5% of trials reported loss to follow-up greater than their fragility index. Trial sample size was positively correlated, and reported p value was negatively correlated, with fragility index.
Conclusions: In critical care trials reporting statistically significant effects on mortality, the findings often depend on a small number of events. Critical care clinicians should be wary of basing decisions on trials with a low fragility index. We advocate the reporting of fragility index for future trials in critical care to aid interpretation and decision making by clinicians.