Similar land-use types usually have similar soil properties, and, most likely, similar microbial communities. Here, we assessed whether land-use types or soil chemical properties are the primary drivers of soil microbial community composition, and how changes in one part of the ecosystem affect another. We applied Ion Torrent sequencing to the bacterial and fungal communities of five different land-use (vegetation) types in the Loess Plateau of China. We found that the overall trend of soil quality was natural forest > plantation > bare land. Dominant bacterial phyla consisted of Proteobacteria (42.35%), Actinobacteria (15.61%), Acidobacteria (13.32%), Bacteroidetes (8.43%), and Gemmatimonadetes (6.0%). The dominant fungi phyla were Ascomycota (40.39%), Basidiomycota (38.01%), and Zygomycota (16.86%). The results of Canonical Correspondence Analysis (CCA) and Redundancy Analysis (RDA) based on land-use types displayed groups according to the land-use types. Furthermore, the bacterial communities were mainly organized by soil organic carbon (SOC). The fungal communities were mainly related to available phosphorus (P). The results suggested that the changes of land use type generated changes in soil chemical properties, controlling the composition of microbial community in the semiarid Loess Plateau region. The microbial community could be an indicator for soil quality with respect to ecological restoration.