Previous statistical voxelwise lesion-behavior mapping (VLBM) studies have demonstrated that spatial neglect is associated with cortical and subcortical gray matter damage. However, it has also been suggested that the disorder may result from white matter injury. Our aim was to investigate the white matter connectivity in a large sample of 140 stroke patients. We combined a VLBM approach with the histological maps of the human white matter fiber tracts provided by the Jülich probabilistic cytoarchitectonic atlas. We found that damage of right perisylvian white matter connections-the superior longitudinal fasciculus, the inferior occipitofrontal fasciculus, and the superior occipitofrontal fasciculus-is a typical finding in patients with spatial neglect. However, the analysis also revealed that the largest portion of the lesion area, namely between 89.1% and 96.6%, affected brain structures other than the perisylvian white matter fiber tracts. Predominantly, these included gray matter structures such as the superior temporal, inferior parietal, inferior frontal, and insular cortices, as well as subcortically the putamen and the caudate nucleus. Damage of gray matter structures thus appears to be a strong predictor of spatial neglect.