We have recently identified transcription factors (TFs) that are key drivers of breast cancer risk. To better understand the pathways or sub-networks in which these TFs mediate their function we sought to identify upstream modulators of their activity. We applied the MINDy (Modulator Inference by Network Dynamics) algorithm to four TFs (ESR1, FOXA1, GATA3 and SPDEF) that are key drivers of estrogen receptor-positive (ER+) breast cancer risk, as well as cancer progression. Our computational analysis identified over 500 potential modulators. We assayed 189 of these and identified 55 genes with functional characteristics that were consistent with a role as TF modulators. In the future, the identified modulators may be tested as potential therapeutic targets, able to alter the activity of TFs that are critical in the development of breast cancer.