The complexity of building models directly affects the application efficiencies of 3D urban maps. To address the challenges of building models with various structures, we propose a structural simplification method in this paper. The geometric structures of building models are classified into three categories: embedded structures, compositional structures, and connecting structures, which can be extracted separately through convex/concave analysis. Some specific rules are proposed for the simplification of geometric structures, and the building models are suggested to be simplified progressively. The robustness and efficiency of the method are demonstrated by experiments, and the applications of the Levels of detail of the building models are illustrated.