Basal-like breast cancer (BLBC) is a heterogeneous disease with poor prognosis; however, its cellular origins and aetiology are poorly understood. In this study, we show that inhibitor of differentiation 4 (ID4) is a key regulator of mammary stem cell self-renewal and marks a subset of BLBC with a putative mammary basal cell of origin. Using an ID4GFP knock-in reporter mouse and single-cell transcriptomics, we show that ID4 marks a stem cell-enriched subset of the mammary basal cell population. ID4 maintains the mammary stem cell pool by suppressing key factors required for luminal differentiation. Furthermore, ID4 is specifically expressed by a subset of human BLBC that possess a very poor prognosis and a transcriptional signature similar to a mammary stem cell. These studies identify ID4 as a mammary stem cell regulator, deconvolute the heterogeneity of BLBC and link a subset of mammary stem cells to the aetiology of BLBC.