To identify the association between the functional and structural changes of default mode network (DMN) underlying the cognitive impairment in Late-onset depression (LOD), 32 LOD patients and 39 normal controls were recruited and underwent resting-state fMRI, DTI scans, and cognitive assessments. Seed-based correlation analysis was conducted to explore the functional connectivity (FC) of the DMN. Deterministic tractography between FC-impaired regions was performed to examine the structural connectivity (SC). Partial correlation analyses were employed to evaluate the cognitive association of those altered FC and SC. Compared with controls, LOD patients showed decreased FC between DMN and the cingulo-opercular network (CON), as well as the thalamus. Decreased FA and increased RD of these fiber tracts connecting DMN with CON were found in LOD patient. The DMN-CON FC and the FA, RD of the fiber tracts were both significantly correlated with the cognitive performance. Therefore, the cognitive impairment in LOD might be associated with the decreased FC between the DMN and the CON, which probably resulted from the demyelination of the white matter.