A methodological review of recent meta-analyses has found significant heterogeneity in age between randomized groups

Abstract

Background: There is evidence to suggest that component randomized controlled trials (RCTs) within systematic reviews may be biased. It is important that these reviews are identified to prevent erroneous conclusions influencing health care policies and decisions.

Purpose: To assess the likelihood of bias in trials in 12 meta-analyses.

Design: A review of 12 systematic reviews.

Data sources: Twelve recently published systematic reviews with 503 component randomized trials, published in the British Medical Journal, The Lancet, Journal of the American Medical Association, and The Annals of Internal Medicine before May 2012.

Study selection and data extraction: Systematic reviews were eligible for inclusion if they included only RCTs. We obtained the full text for the component RCTs of the 12 systematic reviews (in English only). We extracted summary data on age, number of participants in each treatment group, and the method of allocation concealment for each RCT.

Data synthesis: Five of the 12 meta-analyses exhibited heterogeneity in age differences (I(2) > 0.30), when there should have been none. In two meta-analyses, the age of the intervention group was significantly greater than that of the control group. Inadequate allocation concealment was a statistically significant predictor of heterogeneity in one trial as observed by a metaregression.

Conclusions: Most of the sample of recent meta-analyses showed that there were signs of imbalance and/or heterogeneity in ages between treatment groups, when there should have been none. Systematic reviewers might consider using the techniques described here to assess the validity of their findings.

Keywords: Heterogeneity; Meta-analysis; Methods; Randomized controlled trials; Selection bias; Systematic review.