Objectives: To determine which factors predict favorable results and positive conclusions in systematic reviews (SRs) and to assess the level of agreement between SR results and conclusions.
Study design and setting: A sample of 296 English SRs indexed in MEDLINE (November, 2004) was obtained. Two investigators independently categorized SR characteristics, results, and conclusions. Descriptive analyses and logistic regression predicting favorable results (nonstatistically significant and statistically significant positive) and positive conclusions were conducted. The level of concordance between results and conclusions was assessed using a weighted-kappa statistic.
Results: Overall, 36.5% of the SRs had favorable results, increasing to 57.7% for Cochrane and 64.3% for non-Cochrane reviews with a meta-analysis of the primary outcome. Non-Cochrane reviews with a meta-analysis of the primary outcome were twice as likely to have positive conclusions as Cochrane reviews with such an analysis (P-value<0.05). The weighted kappa for agreement between SR results and conclusions was 0.55. It was lower for Cochrane (0.41) vs. non-Cochrane (0.67) reviews.
Conclusion: SRs including a meta-analysis of the primary outcome may be affected by indirect publication bias in our sample. Differences between the results and conclusions of Cochrane and non-Cochrane reviews were apparent. Further research on publication-related issues of SRs is warranted.