Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3/BP4 criteria

Vikas Pejaver; Alicia B Byrne; Bing-Jian Feng; Kymberleigh A Pagel; Sean D Mooney; Rachel Karchin; Anne O'Donnell-Luria; Steven M Harrison; Sean V Tavtigian; Marc S Greenblatt; Leslie G Biesecker; Predrag Radivojac; Steven E Brenner; ClinGen Sequence Variant Interpretation Working Group

doi:10.1016/j.ajhg.2022.10.013

Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3/BP4 criteria

Am J Hum Genet. 2022 Dec 1;109(12):2163-2177. doi: 10.1016/j.ajhg.2022.10.013. Epub 2022 Nov 21.

Collaborators

ClinGen Sequence Variant Interpretation Working Group:
Leslie G Biesecker, Steven M Harrison, Ahmad A Tayoun, Jonathan S Berg, Steven E Brenner, Garry R Cutting, Sian Ellard, Marc S Greenblatt, Peter Kang, Izabela Karbassi, Rachel Karchin, Jessica Mester, Anne O'Donnell-Luria, Tina Pesaran, Sharon E Plon, Heidi L Rehm, Natasha T Strande, Sean V Tavtigian, Scott Topper

Affiliations

¹ Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA 98195, USA.
² Program in Medical and Population Genetics, Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA; Analytic and Translational Genetics Unit, Massachusetts General Hospital, Boston, MA 02114, USA.
³ Department of Dermatology, University of Utah, Salt Lake City, UT 84132, USA; Huntsman Cancer Institute, University of Utah, Salt Lake City, UT 84112, USA.
⁴ The Institute for Computational Medicine, The Johns Hopkins University, Baltimore, MD 21218, USA.
⁵ Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA 98195, USA.
⁶ The Institute for Computational Medicine, The Johns Hopkins University, Baltimore, MD 21218, USA; Departments of Biomedical Engineering, Oncology, and Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA.
⁷ Program in Medical and Population Genetics, Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA; Analytic and Translational Genetics Unit, Massachusetts General Hospital, Boston, MA 02114, USA; Division of Genetics and Genomics, Boston Children's Hospital, Boston, MA 02115, USA.
⁸ Program in Medical and Population Genetics, Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA; Ambry Genetics, Aliso Viejo, CA 92656, USA.
⁹ Department of Oncological Sciences, Huntsman Cancer Institute, University of Utah School of Medicine, Salt Lake City, UT 84112, USA.
¹⁰ Department of Medicine and University of Vermont Cancer Center, University of Vermont, Larner College of Medicine, Burlington, VT 05405, USA.
¹¹ Center for Precision Health Research, National Human Genome Research Institute, National Institutes of Health, Bethesda, MD 20892, USA.
¹² Khoury College of Computer Sciences, Northeastern University, Boston, MA 02115, USA. Electronic address: predrag@northeastern.edu.
¹³ Department of Plant and Microbial Biology and Center for Computational Biology, University of California, Berkeley, Berkeley, CA 94720, USA. Electronic address: brenner@compbio.berkeley.edu.

Abstract

Recommendations from the American College of Medical Genetics and Genomics and the Association for Molecular Pathology (ACMG/AMP) for interpreting sequence variants specify the use of computational predictors as "supporting" level of evidence for pathogenicity or benignity using criteria PP3 and BP4, respectively. However, score intervals defined by tool developers, and ACMG/AMP recommendations that require the consensus of multiple predictors, lack quantitative support. Previously, we described a probabilistic framework that quantified the strengths of evidence (supporting, moderate, strong, very strong) within ACMG/AMP recommendations. We have extended this framework to computational predictors and introduce a new standard that converts a tool's scores to PP3 and BP4 evidence strengths. Our approach is based on estimating the local positive predictive value and can calibrate any computational tool or other continuous-scale evidence on any variant type. We estimate thresholds (score intervals) corresponding to each strength of evidence for pathogenicity and benignity for thirteen missense variant interpretation tools, using carefully assembled independent data sets. Most tools achieved supporting evidence level for both pathogenic and benign classification using newly established thresholds. Multiple tools reached score thresholds justifying moderate and several reached strong evidence levels. One tool reached very strong evidence level for benign classification on some variants. Based on these findings, we provide recommendations for evidence-based revisions of the PP3 and BP4 ACMG/AMP criteria using individual tools and future assessment of computational methods for clinical interpretation.

Keywords: ACMG/AMP recommendations; PP3/BP4 criteria; clinical classification; computational predictors; in silico tools; likelihood ratio; posterior probability; variant interpretation.

Publication types

Research Support, N.I.H., Extramural

MeSH terms

Calibration*
Consensus
Educational Status
Humans
Virulence

Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3/BP4 criteria

Authors

Collaborators

Affiliations

Abstract

Publication types

MeSH terms

Grants and funding