Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review

Daeria O Lawson; Alvin Leenus; Lawrence Mbuagbaw

doi:10.1186/s40814-019-0544-0

Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review

Pilot Feasibility Stud. 2020 Jan 30:6:13. doi: 10.1186/s40814-019-0544-0. eCollection 2020.

Authors

Daeria O Lawson¹, Alvin Leenus², Lawrence Mbuagbaw^{1

3}

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4K1 Canada.
² Faculty of Health Sciences, McMaster University, 1280 Main Street West, Hamilton, ON L8S 4K1 Canada.
³ Biostatistics Unit, Father Sean O'Sullivan Research Centre, St. Joseph's Healthcare Hamilton, Hamilton, ON L8N 4A6 Canada.

Abstract

Background: A relatively novel method of appraisal, methodological reviews (MRs) are used to synthesize information on the methods used in health research. There are currently no guidelines available to inform the reporting of MRs.

Objectives: This pilot review aimed to determine the feasibility of a full review and the need for reporting guidance for methodological reviews.

Methods: Search strategy: We conducted a search of PubMed, restricted to 2017 to include the most recently published studies, using different search terms often used to describe methodological reviews: "literature survey" OR "meta-epidemiologic* review" OR "meta-epidemiologic* survey" OR "methodologic* review" OR "methodologic* survey" OR "systematic survey."Data extraction: Study characteristics including country, nomenclature, number of included studies, search strategy, a priori protocol use, and sampling methods were extracted in duplicate and summarized.Outcomes: Primary feasibility outcomes were the sensitivity and specificity of the search terms (criteria for success of feasibility set at sensitivity and specificity of ≥ 70%).Analysis: The estimates are reported as a point estimate (95% confidence interval).

Results: Two hundred thirty-six articles were retrieved and 31 were included in the final analysis. The most accurate search term was "meta-epidemiological" (sensitivity [Sn] 48.39; 95% CI 31.97-65.16; specificity [Sp] 97.56; 94.42-98.95). The majority of studies were published by authors from Canada (n = 12, 38.7%), and Japan and USA (n = 4, 12.9% each). The median (interquartile range [IQR]) number of included studies in the MRs was 77 (13-1127). Reporting of a search strategy was done in most studies (n = 23, 74.2%). The use of a pre-published protocol (n = 7, 22.6%) or a justifiable sampling method (n = 5, 16.1%) occurred rarely.

Conclusions: Using the MR nomenclature identified, it is feasible to build a comprehensive search strategy and conduct a full review. Given the variation in reporting practices and nomenclature attributed to MRs, there is a need for guidance on standardized and transparent reporting of MRs. Future guideline development would likely include stakeholders from Canada, USA, and Japan.

Keywords: Feasibility; Guidelines; Methodological review; Nomenclature; Pilot; Reporting.