Transparency and reproducibility in the Adolescent Brain Cognitive Development (ABCD) study

Abstract

Background: Transparency can build trust in the scientific process, but scientific findings can be undermined by poor and obscure data use and reporting practices. The purpose of this work is to report how data from the Adolescent Brain Cognitive Development (ABCD) Study has been used to date, and to provide practical recommendations on how to improve the transparency and reproducibility of findings.

Methods: Articles published from 2017 to 2023 that used ABCD Study data were reviewed using more than 30 data extraction items to gather information on data use practices. Total frequencies were reported for each extraction item, along with computation of a Level of Completeness (LOC) score that represented overall endorsement of extraction items. Univariate linear regression models were used to examine the correlation between LOC scores and individual extraction items. Post hoc analysis included examination of whether LOC scores were correlated with the logged 2-year journal impact factor.

Results: There were 549 full-length articles included in the main analysis. Analytic scripts were shared in 30 % of full-length articles. The number of participants excluded due to missing data was reported in 60 % of articles, and information on missing data for individual variables (e.g., household income) was provided in 38 % of articles. A table describing the analytic sample was included in 83 % of articles. A race and/or ethnicity variable was included in 78 % of reviewed articles, while its inclusion was justified in only 41 % of these articles. LOC scores were highly correlated with extraction items related to examination of missing data. A bottom 10 % of LOC score was significantly correlated with a lower logged journal impact factor when compared to the top 10 % of LOC scores (β=-0.77, 95 % -1.02, -0.51; p-value < 0.0001).

Conclusion: These findings highlight opportunities for improvement in future papers using ABCD Study data to readily adapt analytic practices for better transparency and reproducibility efforts. A list of recommendations is provided to facilitate adherence in future research.

Keywords: Adolescent; Best practices; Cognitive; Neuroimaging; Reproducibility; Transparency.