Androgen receptor (AR) is a hormone-activated transcription factor that plays important roles in prostate development and function, as well as malignant transformation. The downstream pathways of AR, however, are incompletely understood. AR has been primarily known as a transcriptional activator inducing prostate-specific gene expression. Through integrative analysis of genome-wide AR occupancy and androgen-regulated gene expression, here we report AR as a globally acting transcriptional repressor. This repression is mediated by androgen-responsive elements (ARE) and dictated by Polycomb group protein EZH2 and repressive chromatin remodeling. In embryonic stem cells, AR-repressed genes are occupied by EZH2 and harbor bivalent H3K4me3 and H3K27me3 modifications that are characteristic of differentiation regulators, the silencing of which maintains the undifferentiated state. Concordantly, these genes are silenced in castration-resistant prostate cancer rendering a stem cell-like lack of differentiation and tumor progression. Collectively, our data reveal an unexpected role of AR as a transcriptional repressor inhibiting non-prostatic differentiation and, upon excessive signaling, resulting in cancerous dedifferentiation.