Histone modifications are fundamental epigenetic regulators that control many crucial cellular processes. However, whether these marks can be passed on from mammalian gametes to the next generation is a long-standing question that remains unanswered. Here, by developing a highly sensitive approach, STAR ChIP-seq, we provide a panoramic view of the landscape of H3K4me3, a histone hallmark for transcription initiation, from developing gametes to post-implantation embryos. We find that upon fertilization, extensive reprogramming occurs on the paternal genome, as H3K4me3 peaks are depleted in zygotes but are readily observed after major zygotic genome activation at the late two-cell stage. On the maternal genome, we unexpectedly find a non-canonical form of H3K4me3 (ncH3K4me3) in full-grown and mature oocytes, which exists as broad peaks at promoters and a large number of distal loci. Such broad H3K4me3 peaks are in contrast to the typical sharp H3K4me3 peaks restricted to CpG-rich regions of promoters. Notably, ncH3K4me3 in oocytes overlaps almost exclusively with partially methylated DNA domains. It is then inherited in pre-implantation embryos, before being erased in the late two-cell embryos, when canonical H3K4me3 starts to be established. The removal of ncH3K4me3 requires zygotic transcription but is independent of DNA replication-mediated passive dilution. Finally, downregulation of H3K4me3 in full-grown oocytes by overexpression of the H3K4me3 demethylase KDM5B is associated with defects in genome silencing. Taken together, these data unveil inheritance and highly dynamic reprogramming of the epigenome in early mammalian development.