Neurodegeneration is associated with increased frequency of neurological soft signs (NSS). We designed the present study to investigate the association between NSS and subjective memory complaints, cognitive function and apolipoprotein E genotype in a community-dwelling sample of volunteers participating in an ongoing longitudinal program investigating predictors of cognitive decline. NSS were found to be associated with apolipoprotein E (APOE) epsilon4 genotype (p = 0.015), age (p = 0.012) and poor cognitive performance, as assessed by the Mini Mental State Examination (p = 0.053). There was no significant difference between subjects with and without memory complaints in relation to the frequency of NSS (p = 0.130). The association with age and the APOE epsilon4 genotype suggests that the systematic investigation of NSS may contribute to identify subjects at risk of clinically significant cognitive decline in later life.