A comparison of methods to harmonize cortical thickness measurements across scanners and sites

Delin Sun; Gopalkumar Rakesh; Courtney C Haswell; Mark Logue; C Lexi Baird; Erin N O'Leary; Andrew S Cotton; Hong Xie; Marijo Tamburrino; Tian Chen; Emily L Dennis; Neda Jahanshad; Lauren E Salminen; Sophia I Thomopoulos; Faisal Rashid; Christopher R K Ching; Saskia B J Koch; Jessie L Frijling; Laura Nawijn; Mirjam van Zuiden; Xi Zhu; Benjamin Suarez-Jimenez; Anika Sierk; Henrik Walter; Antje Manthey; Jennifer S Stevens; Negar Fani; Sanne J H van Rooij; Murray Stein; Jessica Bomyea; Inga K Koerte; Kyle Choi; Steven J A van der Werff; Robert R J M Vermeiren; Julia Herzog; Lauren A M Lebois; Justin T Baker; Elizabeth A Olson; Thomas Straube; Mayuresh S Korgaonkar; Elpiniki Andrew; Ye Zhu; Gen Li; Jonathan Ipser; Anna R Hudson; Matthew Peverill; Kelly Sambrook; Evan Gordon; Lee Baugh; Gina Forster; Raluca M Simons; Jeffrey S Simons; Vincent Magnotta; Adi Maron-Katz; Stefan du Plessis; Seth G Disner; Nicholas Davenport; Daniel W Grupe; Jack B Nitschke; Terri A deRoon-Cassini; Jacklynn M Fitzgerald; John H Krystal; Ifat Levy; Miranda Olff; Dick J Veltman; Li Wang; Yuval Neria; Michael D De Bellis; Tanja Jovanovic; Judith K Daniels; Martha Shenton; Nic J A van de Wee; Christian Schmahl; Milissa L Kaufman; Isabelle M Rosso; Scott R Sponheim; David Bernd Hofmann; Richard A Bryant; Kelene A Fercho; Dan J Stein; Sven C Mueller; Bobak Hosseini; K Luan Phan; Katie A McLaughlin; Richard J Davidson; Christine L Larson; Geoffrey May; Steven M Nelson; Chadi G Abdallah; Hassaan Gomaa; Amit Etkin; Soraya Seedat; Ilan Harpaz-Rotem; Israel Liberzon; Theo G M van Erp; Yann Quidé; Xin Wang; Paul M Thompson; Rajendra A Morey

doi:10.1016/j.neuroimage.2022.119509

A comparison of methods to harmonize cortical thickness measurements across scanners and sites

Neuroimage. 2022 Nov 1:261:119509. doi: 10.1016/j.neuroimage.2022.119509. Epub 2022 Jul 30.

Authors

Delin Sun¹, Gopalkumar Rakesh², Courtney C Haswell², Mark Logue³, C Lexi Baird², Erin N O'Leary⁴, Andrew S Cotton⁴, Hong Xie⁴, Marijo Tamburrino⁴, Tian Chen⁵, Emily L Dennis⁶, Neda Jahanshad⁷, Lauren E Salminen⁷, Sophia I Thomopoulos⁷, Faisal Rashid⁷, Christopher R K Ching⁷, Saskia B J Koch⁸, Jessie L Frijling⁹, Laura Nawijn¹⁰, Mirjam van Zuiden⁹, Xi Zhu¹¹, Benjamin Suarez-Jimenez¹², Anika Sierk¹³, Henrik Walter¹³, Antje Manthey¹³, Jennifer S Stevens¹⁴, Negar Fani¹⁴, Sanne J H van Rooij¹⁴, Murray Stein¹⁵, Jessica Bomyea¹⁵, Inga K Koerte¹⁶, Kyle Choi¹⁷, Steven J A van der Werff¹⁸, Robert R J M Vermeiren¹⁹, Julia Herzog²⁰, Lauren A M Lebois²¹, Justin T Baker²², Elizabeth A Olson²³, Thomas Straube²⁴, Mayuresh S Korgaonkar²⁵, Elpiniki Andrew²⁶, Ye Zhu²⁷, Gen Li²⁷, Jonathan Ipser²⁸, Anna R Hudson²⁹, Matthew Peverill³⁰, Kelly Sambrook³¹, Evan Gordon³², Lee Baugh³³, Gina Forster³⁴, Raluca M Simons³⁵, Jeffrey S Simons³⁶, Vincent Magnotta³⁷, Adi Maron-Katz³⁸, Stefan du Plessis³⁹, Seth G Disner⁴⁰, Nicholas Davenport⁴⁰, Daniel W Grupe⁴¹, Jack B Nitschke⁴², Terri A deRoon-Cassini⁴³, Jacklynn M Fitzgerald⁴⁴, John H Krystal⁴⁵, Ifat Levy⁴⁵, Miranda Olff⁴⁶, Dick J Veltman⁴⁷, Li Wang²⁷, Yuval Neria¹¹, Michael D De Bellis⁴⁸, Tanja Jovanovic⁴⁹, Judith K Daniels⁵⁰, Martha Shenton⁵¹, Nic J A van de Wee¹⁸, Christian Schmahl²⁰, Milissa L Kaufman⁵², Isabelle M Rosso²³, Scott R Sponheim⁴⁰, David Bernd Hofmann²⁴, Richard A Bryant⁵³, Kelene A Fercho⁵⁴, Dan J Stein²⁸, Sven C Mueller²⁹, Bobak Hosseini⁵⁵, K Luan Phan⁵⁶, Katie A McLaughlin⁵⁷, Richard J Davidson⁵⁸, Christine L Larson⁵⁹, Geoffrey May⁶⁰, Steven M Nelson⁶⁰, Chadi G Abdallah⁴⁵, Hassaan Gomaa⁶¹, Amit Etkin⁶², Soraya Seedat³⁹, Ilan Harpaz-Rotem⁴⁵, Israel Liberzon⁶³, Theo G M van Erp⁶⁴, Yann Quidé⁶⁵, Xin Wang⁶⁶, Paul M Thompson⁷, Rajendra A Morey⁶⁷

Affiliations

¹ Brain Imaging and Analysis Center, Duke University, Durham, NC, USA.; Department of Veteran Affairs (VA) Mid-Atlantic Mental Illness Research, Education and Clinical Center, Durham, NC, USA.; Department of Psychology, The Education University of Hong Kong, Hong Kong, China.
² Brain Imaging and Analysis Center, Duke University, Durham, NC, USA.; Department of Veteran Affairs (VA) Mid-Atlantic Mental Illness Research, Education and Clinical Center, Durham, NC, USA.
³ National Center for PTSD, VA Boston Healthcare System, Boston, MA, USA.; Department of Psychiatry, Boston University School of Medicine, Boston, MA, USA.; Biomedical Genetics, Boston University School of Medicine, Boston, MA, USA.; Department of Biostatistics, Boston University School of Public Health, Boston, MA, USA.
⁴ Department of Psychiatry, University of Toledo, Toledo, OH, USA.
⁵ Department of Psychiatry, University of Toledo, Toledo, OH, USA.; Psychiatry Neuroimaging Laboratory, Brigham & Women's Hospital, Boston, MA, USA.
⁶ Psychiatry Neuroimaging Laboratory, Brigham & Women's Hospital, Boston, MA, USA.; Imaging Genetics Center, Mark & Mary Stevens Neuroimaging & Informatics Institute, Keck School of Medicine of USC, Marina del Rey, CA, USA.; Department of Neurology, University of Utah, Salt Lake City, UT, USA.; Stanford Neurodevelopment, Affect, and Psychopathology Laboratory, Stanford, CA, USA.
⁷ Imaging Genetics Center, Mark & Mary Stevens Neuroimaging & Informatics Institute, Keck School of Medicine of USC, Marina del Rey, CA, USA.
⁸ Department of Psychiatry, Amsterdam University Medical Centers, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands.; Donders Institute for Brain, Cognition and Behavior, Centre for Cognitive Neuroimaging, Radboud University Nijmegen, Nijmegen, The Netherlands.
⁹ Department of Psychiatry, Amsterdam University Medical Centers, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands.
¹⁰ Department of Psychiatry, Amsterdam University Medical Centers, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands.; Department of Psychiatry, Amsterdam University Medical Centers, VU University Medical Center, VU University, Amsterdam, The Netherlands.
¹¹ Department of Psychiatry, Columbia University Medical Center, New York, NY, USA.; New York State Psychiatric Institute, New York, NY, USA.
¹² Del Monte Institute for Neuroscience, University of Rochester Medical Center, Rochester, NY, USA.; Department of Psychiatry, Columbia University Medical Center, New York, NY, USA.; New York State Psychiatric Institute, New York, NY, USA.
¹³ University Medical Centre Charité, Berlin, Germany.
¹⁴ Department of Psychiatry and Behavioral Sciences, Emory University School of Medicine, Atlanta, GA, USA.
¹⁵ Department of Psychiatry, University of California San Diego, La Jolla, CA, USA.
¹⁶ Psychiatry Neuroimaging Laboratory, Brigham & Women's Hospital, Boston, MA, USA.; Department of Child and Adolescent Psychiatry, Psychosomatics, and Psychotherapy, Ludwig-Maximilians-Universität, Munich, Germany.
¹⁷ Health Services Research Center, University of California, San Diego, La Jolla, CA, USA.
¹⁸ Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands.; Leiden Institute for Brain and Cognition, Leiden, The Netherlands.
¹⁹ Department of Psychiatry, Leiden University Medical Center, Leiden, The Netherlands.
²⁰ Department of Psychosomatic Medicine and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Heidelberg, Germany.
²¹ Department of Psychiatry, Harvard Medical School, Boston, MA, USA.; Division of Depression and Anxiety Disorders, McLean Hospital, Belmont, MA, USA.
²² Institute for Technology in Psychiatry, McLean Hospital, Harvard University, Belmont, MA, USA.
²³ Department of Psychiatry, Harvard Medical School, Boston, MA, USA.; Center for Depression, Anxiety, and Stress Research, McLean Hospital, Belmont, MA, USA.
²⁴ Institute of Medical Psychology and Systems Neuroscience, University of Münster, Münster, Germany.
²⁵ Brain Dynamics Centre, Westmead Institute of Medical Research, University of Sydney, Westmead, NSW, Australia.
²⁶ Department of Psychology, University of Sydney, Westmead, NSW, Australia.
²⁷ Laboratory for Traumatic Stress Studies, Chinese Academy of Sciences Key Laboratory of Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China.; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China.
²⁸ SA MRC Unit on Risk & Resilience in Mental Disorders, Department of Psychiatry and Neuroscience Institute, University of Cape Town, Cape Town, South Africa.
²⁹ Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium.
³⁰ Department of Psychology, University of Washington, Seattle, WA, USA.
³¹ Department of Radiology, University of Washington, Seattle, WA, USA.
³² Department of Radiology, Washington University, St. Louis, MO, USA.
³³ Division of Basic Biomedical Sciences, Sanford School of Medicine, University of South Dakota, Vermillion, SD, USA.; Center for Brain and Behavior Research, University of South Dakota, Vermillion, SD, USA.; Sioux Falls VA Health Care System, Sioux Falls, SD, USA.
³⁴ Division of Basic Biomedical Sciences, Sanford School of Medicine, University of South Dakota, Vermillion, SD, USA.; Center for Brain and Behavior Research, University of South Dakota, Vermillion, SD, USA.; Brain Health Research Centre, Department of Anatomy, University of Otago, Dunedin, New Zealand.
³⁵ Center for Brain and Behavior Research, University of South Dakota, Vermillion, SD, USA.; Department of Psychology, University of South Dakota, Vermillion, SD, USA.
³⁶ Sioux Falls VA Health Care System, Sioux Falls, SD, USA.; Department of Psychology, University of South Dakota, Vermillion, SD, USA.
³⁷ Department of Radiology, Psychiatry, and Biomedical Engineering, University of Iowa, Iowa City, IA, USA.
³⁸ Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA.
³⁹ Department of Psychiatry, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa.
⁴⁰ Minneapolis VA Health Care System, Minneapolis, MN, USA.; Department of Psychiatry, University of Minnesota, Minneapolis, MN, USA.
⁴¹ Center for Healthy Minds, University of Wisconsin-Madison, Madison, WI, USA.
⁴² Department of Psychiatry, University of Wisconsin-Madison, Madison, WI, USA.
⁴³ Department of Surgery, Division of Trauma and Acute Care Surgery, Medical College of Wisconsin, Milwaukee, WI, USA.
⁴⁴ Department of Psychology, Marquette University, Milwaukee, WI, USA.
⁴⁵ Division of Clinical Neuroscience, National Center for PTSD, West Haven, CT, USA.; Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA.
⁴⁶ Department of Psychiatry, Amsterdam University Medical Centers, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands.; ARQ National Psychotrauma Centre, Diemen, The Netherlands.
⁴⁷ Department of Psychiatry, Amsterdam University Medical Center, location VUMC, Amsterdam, The Netherlands.
⁴⁸ Healthy Childhood Brain Development Developmental Traumatology Research Program, Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, USA.
⁴⁹ Department of Psychiatry and Behavioral Neurosciences, Wayne State University School of Medicine, Detroit, MI, USA.
⁵⁰ Department of Clinical Psychology, University of Groningen, Groningen, The Netherlands.
⁵¹ Psychiatry Neuroimaging Laboratory, Brigham & Women's Hospital, Boston, MA, USA.; VA Boston Healthcare System, Brockton Division, Brockton, MA, USA.
⁵² Department of Psychiatry, Harvard Medical School, Boston, MA, USA.; Division of Women's Mental Health, McLean Hospital, Belmont, MA, USA.
⁵³ School of Psychology, University of New South Wales, Sydney, NSW, Australia.
⁵⁴ Division of Basic Biomedical Sciences, Sanford School of Medicine, University of South Dakota, Vermillion, SD, USA.; Center for Brain and Behavior Research, University of South Dakota, Vermillion, SD, USA.; Sioux Falls VA Health Care System, Sioux Falls, SD, USA.; Civil Aerospace Medical Institute, US Federal Aviation Administration, Oklahoma City, OK, USA.
⁵⁵ Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, USA.
⁵⁶ Department of Psychiatry, University of Illinois at Chicago, Chicago, IL, USA.; Mental Health Service Line, Jesse Brown VA Chicago Health Care System, Chicago, IL, USA.
⁵⁷ Department of Psychology, Harvard University, Cambridge, MA, USA.
⁵⁸ Center for Healthy Minds, University of Wisconsin-Madison, Madison, WI, USA.; Department of Psychiatry, University of Wisconsin-Madison, Madison, WI, USA.; Department of Psychology, University of Wisconsin-Madison, Madison, WI, USA.
⁵⁹ Department of Psychology, University of Wisconsin-Milwaukee, Milwaukee, WI, USA.
⁶⁰ Veterans Integrated Service Network-17 Center of Excellence for Research on Returning War Veterans, Waco, TX, USA.; Department of Psychology and Neuroscience, Baylor University, Waco, TX, USA.; Center for Vital Longevity, School of Behavioral and Brain Sciences, University of Texas at Dallas, Dallas, TX, USA.; Department of Psychiatry and Behavioral Science, Texas A&M University Health Science Center, Bryan, TX, USA.
⁶¹ Department of Psychiatry and Behavioral Health, Pennsylvania State University, Hershey, PA, USA.
⁶² Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA.; VA Palo Alto Health Care System, Palo Alto, CA, USA.
⁶³ Department of Psychiatry and Behavioral Science, Texas A&M University, College Station, TX, USA.
⁶⁴ Clinical Translational Neuroscience Laboratory, Department of Psychiatry and Human Behavior, University of California Irvine, Irvine, CA, USA.; Center for the Neurobiology of Learning and Memory, University of California, Irvine, Irvine, CA, USA.
⁶⁵ School of Psychology, The University of New South Wales, Sydney, NSW, Australia.; Neuroscience Research Australia, Randwick, NSW, Australia.
⁶⁶ Department of Mathematics and Statistics, University of Toledo, Toledo, OH, USA.
⁶⁷ Brain Imaging and Analysis Center, Duke University, Durham, NC, USA.; Department of Veteran Affairs (VA) Mid-Atlantic Mental Illness Research, Education and Clinical Center, Durham, NC, USA.. Electronic address: rajendra.morey@duke.edu.

Abstract

Results of neuroimaging datasets aggregated from multiple sites may be biased by site-specific profiles in participants' demographic and clinical characteristics, as well as MRI acquisition protocols and scanning platforms. We compared the impact of four different harmonization methods on results obtained from analyses of cortical thickness data: (1) linear mixed-effects model (LME) that models site-specific random intercepts (LME_INT), (2) LME that models both site-specific random intercepts and age-related random slopes (LME_INT+SLP), (3) ComBat, and (4) ComBat with a generalized additive model (ComBat-GAM). Our test case for comparing harmonization methods was cortical thickness data aggregated from 29 sites, which included 1,340 cases with posttraumatic stress disorder (PTSD) (6.2-81.8 years old) and 2,057 trauma-exposed controls without PTSD (6.3-85.2 years old). We found that, compared to the other data harmonization methods, data processed with ComBat-GAM was more sensitive to the detection of significant case-control differences (Χ²(3) = 63.704, p < 0.001) as well as case-control differences in age-related cortical thinning (Χ²(3) = 12.082, p = 0.007). Both ComBat and ComBat-GAM outperformed LME methods in detecting sex differences (Χ²(3) = 9.114, p = 0.028) in regional cortical thickness. ComBat-GAM also led to stronger estimates of age-related declines in cortical thickness (corrected p-values < 0.001), stronger estimates of case-related cortical thickness reduction (corrected p-values < 0.001), weaker estimates of age-related declines in cortical thickness in cases than controls (corrected p-values < 0.001), stronger estimates of cortical thickness reduction in females than males (corrected p-values < 0.001), and stronger estimates of cortical thickness reduction in females relative to males in cases than controls (corrected p-values < 0.001). Our results support the use of ComBat-GAM to minimize confounds and increase statistical power when harmonizing data with non-linear effects, and the use of either ComBat or ComBat-GAM for harmonizing data with linear effects.

Keywords: ComBat; ComBat-GAM; Cortical Thickness; Data Harmonization; General Additive Model; Linear Mixed-Effects Model; PTSD; Scanner Effects; Site Effects.

Publication types

Research Support, U.S. Gov't, Non-P.H.S.
Research Support, Non-U.S. Gov't
Research Support, N.I.H., Extramural

MeSH terms

Adolescent
Adult
Aged
Aged, 80 and over
Case-Control Studies
Child
Female
Humans
Magnetic Resonance Imaging* / methods
Male
Middle Aged
Neuroimaging
Stress Disorders, Post-Traumatic*
Young Adult

Abstract

Publication types

MeSH terms

Grants and funding