Towards reproducible brain-wide association studies

S Marek, B Tervo-Clemmens, FJ Calabro, DF Montez… - BioRxiv, 2020 - biorxiv.org
Magnetic resonance imaging (MRI) continues to drive many important neuroscientific
advances. However, progress in uncovering reproducible associations between individual
differences in brain structure/function and behavioral phenotypes (eg, cognition, mental
health) may have been undermined by typical neuroimaging sample sizes (median N= 25),.
Leveraging the Adolescent Brain Cognitive Development (ABCD) Study (N= 11,878), we
estimated the effect sizes and reproducibility of these brain-wide associations studies …
Abstract
Magnetic resonance imaging (MRI) continues to drive many important neuroscientific advances. However, progress in uncovering reproducible associations between individual differences in brain structure/function and behavioral phenotypes (e.g., cognition, mental health) may have been undermined by typical neuroimaging sample sizes (median N=25),. Leveraging the Adolescent Brain Cognitive Development (ABCD) Study (N=11,878), we estimated the effect sizes and reproducibility of these brain-wide associations studies (BWAS) as a function of sample size. The very largest, replicable brain-wide associations for univariate and multivariate methods were r=0.14 and r=0.34, respectively. In smaller samples, typical for brain-wide association studies (BWAS), irreproducible, inflated effect sizes were ubiquitous, no matter the method (univariate, multivariate). Until sample sizes started to approach consortium-levels, BWAS were underpowered and statistical errors assured. Multiple factors contribute to replication failures; here, we show that the pairing of small brain-behavioral phenotype effect sizes with sampling variability is a key element in wide-spread BWAS replication failure. Brain-behavioral phenotype associations stabilize and become more reproducible with sample sizes of N⪆2,000. While investigator-initiated brain-behavior research continues to generate hypotheses and propel innovation, large consortia are needed to usher in a new era of reproducible human brain-wide association studies.
biorxiv.org