Studies that relate change in physical activity to change in health-related quality of life in the general population are needed to confirm associations suggested by cross-sectional studies. In the present study, cross-sectional as well as longitudinal associations between leisure time physical activity and health-related quality of life were studied in an apparently healthy population. The present study showed cross-sectional associations between at least moderately intense leisure time physical activity and general health perceptions, vitality, physical functioning and role limitations due to physical health problems. No associations were present for total leisure time physical activity. Change in leisure time physical activity was associated with change in social functioning in men as well as in women, irrespective of the intensity of physical activity. Only in men, change in total leisure time physical activity was associated with change in vitality and general mental health. In our study, cross-sectional associations were not confirmed by longitudinal analyses. Cross-sectional associations were mainly found for physical components of health-related quality of life, whereas longitudinal associations were predominantly observed for mental components of health-related quality of life. Confirmation of our results by those of other studies is needed in order to quantify health promotion messages.