Shifts in social structure and cultural practices can potentially promote unusual combinations of allele frequencies that drive the evolution of genetic and phenotypic novelties during human evolution. These cultural practices act in combination with geographical and linguistic barriers and can promote faster evolutionary changes shaped by gene-culture interactions. However, specific cases indicative of this interaction are scarce. Here we show that quantitative genetic parameters obtained from cephalometric data taken on 1,203 individuals analyzed in combination with genetic, climatic, social, and life-history data belonging to six South Amerindian populations are compatible with a scenario of rapid genetic and phenotypic evolution, probably mediated by cultural shifts. We found that the Xavánte experienced a remarkable pace of evolution: the rate of morphological change is far greater than expected for its time of split from their sister group, the Kayapó, which occurred around 1,500 y ago. We also suggest that this rapid differentiation was possible because of strong social-organization differences. Our results demonstrate how human groups deriving from a recent common ancestor can experience variable paces of phenotypic divergence, probably as a response to different cultural or social determinants. We suggest that assembling composite databases involving cultural and biological data will be of key importance to unravel cases of evolution modulated by the cultural environment.