Systematic reviews do not adequately report or address missing outcome data in their analyses: a methodological survey

Lara A Kahale; Batoul Diab; Romina Brignardello-Petersen; Arnav Agarwal; Reem A Mustafa; Joey Kwong; Ignacio Neumann; Ling Li; Luciane Cruz Lopes; Matthias Briel; Jason W Busse; Alfonso Iorio; Per Olav Vandvik; Paul Elias Alexander; Gordon Guyatt; Elie A Akl

doi:10.1016/j.jclinepi.2018.02.016

Systematic reviews do not adequately report or address missing outcome data in their analyses: a methodological survey

J Clin Epidemiol. 2018 Jul:99:14-23. doi: 10.1016/j.jclinepi.2018.02.016. Epub 2018 Mar 2.

Authors

Lara A Kahale¹, Batoul Diab¹, Romina Brignardello-Petersen², Arnav Agarwal³, Reem A Mustafa⁴, Joey Kwong⁵, Ignacio Neumann⁶, Ling Li⁷, Luciane Cruz Lopes⁸, Matthias Briel⁹, Jason W Busse¹⁰, Alfonso Iorio¹¹, Per Olav Vandvik¹², Paul Elias Alexander¹³, Gordon Guyatt¹¹, Elie A Akl¹⁴

Affiliations

¹ Clinical Epidemiology Unit, American University of Beirut, Beirut, Lebanon.
² Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Evidence Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Santiago, Chile.
³ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
⁴ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, University of Missouri-Kansas City, Kansas City, MO, USA; Department of Biomedical and Health Informatics, University of Missouri-Kansas City, Kansas City, MO, USA.
⁵ JC School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong, China.
⁶ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Internal Medicine, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile.
⁷ Clinical Research and Evaluation Unit, Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu, China.
⁸ Pharmaceutical Sciences Post Graduate Course, University of Sorocaba, UNISO, Sorocaba, Sao Paulo, Brazil.
⁹ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, University of Basel, Spitalstrasse 12 4031, Basel, Switzerland.
¹⁰ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Anesthesia, McMaster University, Hamilton, Canada; The Michael G. DeGroote Institute for Pain Research and Care, McMaster University, Hamilton, Canada.
¹¹ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Canada.
¹² Norwegian Knowledge Centre for the Health Services, Oslo, Norway; Department of Medicine, Innlandet Hospital Trust, Gjøvik, Norway.
¹³ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
¹⁴ Clinical Epidemiology Unit, American University of Beirut, Beirut, Lebanon; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada. Electronic address: ea32@aub.edu.lb.

PMID: 29505859
DOI: 10.1016/j.jclinepi.2018.02.016

Abstract

Objectives: To describe how systematic review authors report and address categories of participants with potential missing outcome data of trial participants.

Study design and setting: Methodological survey of systematic reviews reporting a group-level meta-analysis.

Results: We included a random sample of 50 Cochrane and 50 non-Cochrane systematic reviews. Of these, 25 reported in their methods section a plan to consider at least one of the 10 categories of missing outcome data; 42 reported in their results, data for at least one category of missing data. The most reported category in the methods and results sections was "unexplained loss to follow-up" (n = 34 in methods section and n = 6 in the results section). Only 19 reported a method to handle missing data in their primary analyses, which was most often complete case analysis. Few reviews (n = 9) reported in the methods section conducting sensitivity analysis to judge risk of bias associated with missing outcome data at the level of the meta-analysis; and only five of them presented the results of these analyses in the results section.

Conclusion: Most systematic reviews do not explicitly report sufficient information on categories of trial participants with potential missing outcome data or address missing data in their primary analyses.

Keywords: Imputation; Meta-analysis; Missing outcome data; Risk of bias; Systematic reviews; Trials.

Publication types

Research Support, Non-U.S. Gov't
Review

MeSH terms

Bias*
Data Accuracy
Data Interpretation, Statistical
Meta-Analysis as Topic*
Outcome Assessment, Health Care / methods
Outcome Assessment, Health Care / statistics & numerical data*
Systematic Reviews as Topic*