Genome-wide sequencing technologies enable investigations of the structural properties of the genome in various spatial dimensions. Here, we review computational techniques developed to model the three-dimensional genome in single cells versus ensembles of cells and assess their underlying assumptions. We further address approaches to study the spatio-temporal aspects of genome organization from single-cell data.