Face recognition (FR) is an interesting topic in recent pattern recognition investigation. Especially, the accuracy of FR is the foremost concern for practical applications. Linear regression classification (LRC) is one of the most famous and effective methods in the FR area. However, it could perform inaccuracy under variant situations such as few training samples, lighting changes, and partial occlusions. In this paper, we propose a novel classification based on LRC, which is called the Census regression classification (CRC), for improving the recognition performance. There are two contributions in this paper. First, an adaptive confidence factor based on Hamming distance upon the Census correlation is proposed to classify the importance of each pixel from comparing training images and a testing image. Secondly, we join a regularized parameter to control the balance between the bias and the variance of the estimation. In order to substantiate the effectiveness, the AR database is adopted to evaluate the performance. Besides, the well-known face recognition approaches including PCA, LDA, ICA, SVM, SRC, LRC, LDRC, URC, and KLRC are compared with the proposed CRC. In experimental results, it can be separated into two parts. First, we prove that the Census similarity is useful for accuracy improvement under different variations. Moreover, the CRC can perform well under lighting changes and partial occlusions.