The Fragility of Statistically Significant Binary Outcomes for Treating Achilles Tendinopathy: A Systematic Review of Randomized Trials

Omkar S Anaspure; Shiv Patel; Anthony N Baumann; Andrew Newsom; Albert T Anastasio; Annunziato Amendola

doi:10.1177/24730114241300160

The Fragility of Statistically Significant Binary Outcomes for Treating Achilles Tendinopathy: A Systematic Review of Randomized Trials

Foot Ankle Orthop. 2024 Nov 20;9(4):24730114241300160. doi: 10.1177/24730114241300160. eCollection 2024 Oct.

Authors

Omkar S Anaspure¹, Shiv Patel¹, Anthony N Baumann^{2

3}, Andrew Newsom², Albert T Anastasio⁴, Annunziato Amendola⁴

Affiliations

¹ Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
² College of Medicine, Northeast Ohio Medical University, Rootstown, OH, USA.
³ Department of Rehabilitation Services, University Hospitals, Cleveland, OH, USA.
⁴ Department Orthopedic Surgery, Duke University, Durham, NC, USA.

Abstract

Background: Randomized controlled trials (RCTs) are the gold standard for treatment efficacy, but foot and ankle RCTs are often small or inconsistent. The Fragility Index (FI) evaluates the stability of significant findings. This study assessed the fragility of RCT outcomes for Achilles tendon pathology (ATP) interventions.

Methods: This systematic review queried PubMed up to May 14, 2024, for RCTs on ATP interventions. RCTs with significant binary outcomes were included. Two reviewers assessed eligibility, extracted data, calculated FIs, and evaluated risk of bias. Frequency-weighted means were used for narrative synthesis.

Results: Eleven RCTs with 4506 patients (mean cohort size: 409.64 ± 160.54) and a mean age of 36.97 ± 13.51 years (n = 4356; 96.67%) were included, covering 24 binary outcomes. The median FI across all outcomes was 3 (interquartile range 1-4; mean 3.92), indicating that changing the outcome of just a few patients could shift a study's results from statistically significant to nonsignificant. Trials having an FI ≤3 comprised 58.33%. Three outcomes (12.5%) had an FI of zero after recalculating P values using the two-sided Fisher exact test. Half of the outcomes were robust. No RCT reported FIs or adjusted significance for multiple testing. Most studies (81.82%) performed 2 or more statistical tests, with an average of 30.81 ± 41.28 P values reported per study. The overall risk of bias was low in 1 study (9.09%) and moderate in 7 (63.64%). Most studies had low risk of bias in randomization (72.73%) and missing outcome data (90.91%).

Conclusion: The FI assesses the fragility of statistically significant binary results, revealing that many ATP RCTs have fragile outcomes due to small sample sizes. A median FI of 3 means that changing the outcome of 3 patients could shift a study's results from statistically significant to nonsignificant.

Keywords: Achilles; Fragility Index; outcomes; randomized controlled trials; statistical significance.