The highly immunogenic human tumor antigen NY-ESO-1 (ESO) is a target of choice for anti-cancer immune therapy. In this study, we assessed spontaneous antibody (Ab) responses to ESO in a large cohort of patients with primary breast cancer (BC) and addressed the correlation between the presence of anti-ESO Ab, the expression of ESO in the tumors and their characteristics. We found detectable Ab responses to ESO in 1% of the patients. Tumors from patients with circulating Ab to ESO exhibited common characteristics, being mainly hormone receptor (HR)⁻ invasive ductal carcinomas of high grade, including both HER2⁻ and HER2⁺ tumors. In line with these results, we detected ESO expression in 20% of primary HR⁻ BC, including both ESO Ab⁺ and Ab⁻ patients, but not in HR⁺ BC. Interestingly, whereas expression levels in ESO⁺ BC were not significantly different between ESO Ab⁺ and Ab⁻ patients, the former had, in average, significantly higher numbers of tumor-infiltrated lymph nodes, indicating that lymph node invasion may be required for the development of spontaneous anti-tumor immune responses. Thus, the presence of ESO Ab identifies a tumor subtype of HR⁻ (HER2⁻ or HER2⁺) primary BC with frequent ESO expression and, together with the assessment of antigen expression in the tumor, may be instrumental for the selection of patients for whom ESO-based immunotherapy may complement standard therapy.