There are two major strategies for building modeling, i.e. data-driven and modeldriven approaches. Those two methods have their own advantage. For example, as for spherical dome the extraction of structure lines from images or LiDAR in the data-driven approach is not practical and impossible. But it is easy to adopt a parametric model for modeling a spheroid. In the paper, we propose a hybrid system that combines the data-driven and model-driven approaches for building modeling. It fuses both approaches' excellence and compensate for each other to increase the degree of completeness of modeling. The input data is the airborne stereo-imagery with orientation parameters. During building reconstruction, depend on the type of buildings the operator may select either data-driven or model-driven approach. For data-driven, the 3D structure lines have to be measured and we can adopt the Split-Merge-Shape algorithm to reconstruct its detailed rooftop structure, such as gable, hipped, cross gable and cross hipped, etc. For roof type in cylindroids or spheroid, the model-driven approach is suggested that an intermediate parametric model will be utilized and converted to a generalized polyhedral model at the end. In the measurement of 3D structure lines from roof corners and 3D point from rooftop, a naked-eye stereo-measurement GUI without special hardware was designed to accomplish the whole process. Experimental results demonstrate that the proposed scheme can generate most kinds of building models effectively.