Congenital Zika virus (ZIKV) syndrome may cause fetal microcephaly in ~1% of affected newborns. Here, we investigate whether the majority of clinically inapparent newborns might suffer from long-term health impairments not readily visible at birth. Infection of immunocompetent pregnant mice with high-dose ZIKV caused severe offspring phenotypes, such as fetal death, as expected. By contrast, low-dose (LD) maternal ZIKV infection resulted in reduced fetal birth weight but no other obvious phenotypes. Male offspring born to LD ZIKV-infected mothers had increased testosterone (TST) levels and were less likely to survive in utero infection compared to their female littermates. Males also presented an increased number of immature neurons in apical and basal hippocampal dendrites, while female offspring had immature neurons in basal dendrites only. Moreover, male offspring with high but not very high (storm) TST levels were more likely to suffer from learning and memory impairments compared to females. Future studies are required to understand the impact of TST on neuropathological and neurocognitive impairments in later life. In summary, increased sex-specific vigilance is required in countries with high ZIKV prevalence, where impaired neurodevelopment may be camouflaged by a healthy appearance at birth.