High plasma homocysteine (Hcy) levels are associated with increased osteoporotic fracture incidence. However, the mechanism remains unclear. We investigated the effect of Hcy-lowering vitamin B12 and folic acid treatment on bone mineral density (BMD) and calcaneal quantitative ultrasound (QUS) parameters. This randomized, double-blind, placebo-controlled trial included participants aged ≥65 years with plasma Hcy levels between 12 and 50 µmol/L. The intervention comprised 2-year supplementation with either a combination of 500 µg B12, 400 µg folic acid, and 600 IU vitamin D or placebo with 600 IU vitamin D only. In total, 1111 participants underwent repeated dual-energy X-ray assessment and 1165 participants underwent QUS. Femoral neck (FN) BMD, lumbar spine (LS) BMD, calcaneal broadband ultrasound attenuation (BUA), and calcaneal speed of sound (SOS) were assessed. After 2 years, FN-BMD and BUA had significantly decreased, while LS-BMD significantly increased (all p < 0.01) and SOS did not change in either treatment arm. No statistically significant differences between the intervention and placebo group were present for FN-BMD (p = 0.24), LS-BMD (p = 0.16), SOS (p = 0.67), and BUA (p = 0.96). However, exploratory subgroup analyses revealed a small positive effect of the intervention on BUA at follow-up among compliant persons >80 years (estimated marginal mean 64.4 dB/MHz for the intervention group and 61.0 dB/MHz for the placebo group, p = 0.04 for difference). In conclusion, this study showed no overall effect of treatment with vitamin B12 and folic acid on BMD or QUS parameters in elderly, mildly hyperhomocysteinemic persons, but suggests a small beneficial effect on BUA in persons >80 years who were compliant in taking the supplement.