Airborne Light Detection and Ranging (LiDAR) has the ability of acquiring huge and highly accurate point cloud. Therefore, the processing of huge point clouds has become an important research topic and has drawn increasing attention in the fields of remote sensing. In this paper, an automatic three-dimensional (3D) model retrieval system is introduced to retrieve building models using point clouds as input queries. The proposed system includes a novel point cloud encoding approach and a 3D building model database acquired from Internet. The encoding approach is based on low frequency spherical-harmonic basis functions which provide compact representation of 3D data and has the properties of noise insensitivity and rotation invariance. To ease the problem of inconsistent encoding of point cloud and building models, three steps, namely, model resampling, datum determination, and data filling, are introduced in the preprocessing. The origins of input point cloud and models are aligned in the step of datum determination, and the aliasing problems caused by sparse and incomplete sampling of point clouds are eased in the steps of model resampling and data filling. The experimental results show that the proposed method which consistently encodes a point cloud and a model can yield satisfactory retrieval results.