The quality of evidence in Chinese meta-analyses needs to be improved

Abstract

Objectives: We conducted a systematic review using GRADE (Grading of Recommendations Assessment, Development and Evaluation) system to assess the quality of evidence of Chinese meta-analyses (MAs).

Design/setting: A systematic review of MAs listed in Chinese Biomedicine Literature Database from January 2010 to December 2012.

Methods: Mesh term "meta-analysis" was used to search the Chinese Biomedicine Literature Database from January 2010 to December 2012. Characteristics and main outcomes of each included MA were extracted, and the GRADE system was used to assess the quality of evidence for each outcome. A 10% random sample of Cochrane MAs between 2010 and 2012 was also assessed as control group.

Results: A total of 564 Chinese MAs (including 1,237 main outcomes) and 95 Cochrane MAs (including 251 main outcomes) were identified. Almost half (600, 48.5%) of the outcomes in Chinese MAs were rated as low, and the proportion of outcomes with high or moderate quality of evidence was lower in Chinese MAs than Cochrane MAs (Chinese, 406 [32.8%] vs. Cochrane MAs, 155 [61.8%], P < 0.001). Of the outcomes in Chinese MAs, 1,012 (81.8%) were downgraded for risk of bias. Other common factors for downgrading were imprecision (448, 36.2%), publication bias (418, 33.8%), inconsistency (351, 28.4%), and indirectness (1, 0.1%).

Conclusions: Chinese MAs were of low quality of evidence. Risk of bias, inconsistency, and publication bias were the three most common downgrade factors in Chinese MAs. Efforts must be made to improve quality of evidence of Chinese MAs, and a call for more rigorous training of investigators in China is warranted.

Keywords: Cochrane review; GRADE; Meta-analysis (MA); Quality of evidence; Systematic review; TCM review; Traditional Chinese medicine (TCM).