GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines

Carlos A Cuello-Garcia; Nancy Santesso; Rebecca L Morgan; Jos Verbeek; Kris Thayer; Mohammed T Ansari; Joerg Meerpohl; Lukas Schwingshackl; Srinivasa Vittal Katikireddi; Jan L Brozek; Barnaby Reeves; Mohammad H Murad; Maicon Falavigna; Reem Mustafa; Deborah L Regidor; Paul Elias Alexander; Paul Garner; Elie A Akl; Gordon Guyatt; Holger J Schünemann

doi:10.1016/j.jclinepi.2021.11.026

GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines

J Clin Epidemiol. 2022 Feb:142:200-208. doi: 10.1016/j.jclinepi.2021.11.026. Epub 2021 Nov 17.

Authors

Carlos A Cuello-Garcia¹, Nancy Santesso², Rebecca L Morgan², Jos Verbeek³, Kris Thayer⁴, Mohammed T Ansari⁵, Joerg Meerpohl⁶, Lukas Schwingshackl⁷, Srinivasa Vittal Katikireddi⁸, Jan L Brozek⁹, Barnaby Reeves¹⁰, Mohammad H Murad¹¹, Maicon Falavigna¹², Reem Mustafa¹³, Deborah L Regidor¹⁴, Paul Elias Alexander², Paul Garner¹⁵, Elie A Akl¹⁶, Gordon Guyatt⁹, Holger J Schünemann¹⁷

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Health Quality Improvement. Tecnologico de Monterrey School of Medicine, Monterrey, Nuevo Leon, Mexico. Electronic address: cuelloca@mcmaster.ca.
² Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
³ Cochrane Work Review Group. Occupational Medicine. Finnish Institute of Occupational Health, Helsinki, Uusimaa, Finland.
⁴ Integrated Risk Information System (IRIS) Division, National Center for Environmental Assessment, Environmental Protection Agency, Washington, D.C., USA.
⁵ School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, QC, Canada.
⁶ Institute for Evidence in Medicine, Medical Center & Faculty of Medicine, University of Freiburg, Freiburg, Baden-Wurttemberg, Germany; Cochrane Germany, Cochrane Germany Foundation, Freiburg, Baden-Wurttemberg, Germany.
⁷ Institute for Evidence in Medicine, Medical Center & Faculty of Medicine, University of Freiburg, Freiburg, Baden-Wurttemberg, Germany.
⁸ MRC/CSO Social & Public Health Sciences Unit, University of Glasgow, Greater Glasgow, Scotland.
⁹ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University. Hamilton, Ontario, Canada.
¹⁰ School of Clinical Sciences, University of Bristol, Bristol, South West, England.
¹¹ Evidence-Based Practice Center, Mayo Clinic, Rochester, Minnesota, USA.
¹² Federal University of Rio Grande do Sul, Institute for Health Technology Assessment, Porto Alegre, Rio Grande do Sul, Brazil.
¹³ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Division of Nephrology and Hypertension, Department of Medicine, University of Kansas Medical Center, Kansas City, Missouri USA.
¹⁴ Evidence Services, Kaiser Permanente, Care Management Institute, Oakland, California, USA.
¹⁵ Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, Merseyside, England.
¹⁶ Department of Internal Medicine, American University of Beirut, Beirut Governorate, Lebanon.
¹⁷ Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University. Hamilton, Ontario, Canada. Electronic address: schuneh@mcmaster.ca.

Abstract

Background and objective: This is the 24th in the ongoing series of articles describing the GRADE approach for assessing the certainty of a body of evidence in systematic reviews and health technology assessments and how to move from evidence to recommendations in guidelines.

Methods: Guideline developers and authors of systematic reviews and other evidence syntheses use randomized controlled studies (RCTs) and non-randomized studies of interventions (NRSI) as sources of evidence for questions about health interventions. RCTs with low risk of bias are the most trustworthy source of evidence for estimating relative effects of interventions because of protection against confounding and other biases. However, in several instances, NRSI can still provide valuable information as complementary, sequential, or replacement evidence for RCTs.

Results: In this article we offer guidance on the decision regarding when to search for and include either or both types of studies in systematic reviews to inform health recommendations.

Conclusion: This work aims to help methodologists in review teams, technology assessors, guideline panelists, and anyone conducting evidence syntheses using GRADE.

Keywords: Certainty of evidence; GRADE; Non-randomized studies; Quality of evidence; ROBINS; Risk of bias.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Bias
Humans
Research Design*
Systematic Reviews as Topic
Technology Assessment, Biomedical*

Abstract

Publication types

MeSH terms

Grants and funding