The limitations of simple gene set enrichment analysis assuming gene independence

Pablo Tamayo; George Steinhardt; Arthur Liberzon; Jill P Mesirov

doi:10.1177/0962280212460441

The limitations of simple gene set enrichment analysis assuming gene independence

Stat Methods Med Res. 2016 Feb;25(1):472-87. doi: 10.1177/0962280212460441. Epub 2012 Oct 14.

Authors

Pablo Tamayo¹, George Steinhardt², Arthur Liberzon³, Jill P Mesirov⁴

Affiliations

¹ The Eli and Edythe L. Broad Institute of Massachusetts Institute of Technology and Harvard University, Cambridge, MA, USA tamayo@broadinstitute.org.
² Boston University Bioinformatics Program, Boston University, Boston, MA, USA.
³ The Eli and Edythe L. Broad Institute of Massachusetts Institute of Technology and Harvard University, Cambridge, MA, USA.
⁴ The Eli and Edythe L. Broad Institute of Massachusetts Institute of Technology and Harvard University, Cambridge, MA, USA Boston University Bioinformatics Program, Boston University, Boston, MA, USA.

Abstract

Since its first publication in 2003, the Gene Set Enrichment Analysis method, based on the Kolmogorov-Smirnov statistic, has been heavily used, modified, and also questioned. Recently a simplified approach using a one-sample t-test score to assess enrichment and ignoring gene-gene correlations was proposed by Irizarry et al. 2009 as a serious contender. The argument criticizes Gene Set Enrichment Analysis's nonparametric nature and its use of an empirical null distribution as unnecessary and hard to compute. We refute these claims by careful consideration of the assumptions of the simplified method and its results, including a comparison with Gene Set Enrichment Analysis's on a large benchmark set of 50 datasets. Our results provide strong empirical evidence that gene-gene correlations cannot be ignored due to the significant variance inflation they produced on the enrichment scores and should be taken into account when estimating gene set enrichment significance. In addition, we discuss the challenges that the complex correlation structure and multi-modality of gene sets pose more generally for gene set enrichment methods.

Keywords: Gene set enrichment analysis; gene expression.

Publication types

Research Support, N.I.H., Extramural

MeSH terms

Biostatistics
Databases, Genetic / statistics & numerical data
Epistasis, Genetic
Gene Expression Profiling / statistics & numerical data
Genome, Human*
Humans
Knowledge Bases*
Models, Statistical
Oligonucleotide Array Sequence Analysis / statistics & numerical data
Statistics, Nonparametric

Abstract

Publication types

MeSH terms

Grants and funding