This study examined the association between smoking, physical activity and dietary choice at 36 and 43 years, and change in these lifestyle behaviors between these ages, and decline in verbal memory and visual search speed between 43 and 60-64 years in 1018 participants from MRC National Survey of Health and Development (NSHD, the British 1946 birth cohort). ANCOVA models were adjusted for sex, social class of origin, childhood cognition, educational attainment, adult social class, and depression; then the lifestyle behaviors were additionally mutually adjusted. Results showed that healthy dietary choice and physical activity were associated, respectively, with slower memory and visual search speed decline over 20 years, with evidence that increasing physical activity was important. Adopting positive health behaviors from early midlife may be beneficial in reducing the rate of cognitive decline and ultimately reducing the risk of dementia.