Facial expression recognition using optimized active regions

In this paper, we report an effective facial expression recognition system for classifying six or seven basic expressions accurately. Instead of using the whole face region, we define three kinds of active regions, i.e., left eye regions, right eye regions and mouth regions. We propose a method to search optimized active regions from the three kinds of active regions. A Convolutional Neural Network (CNN) is trained for each kind of optimized active regions to extract features and classify expressions. In order to get representable features, histogram equalization, rotation correction and spatial normalization are carried out on the expression images. A decision-level fusion method is applied, by which the final result of expression recognition is obtained via majority voting of the three CNNs’ results. Experiments on both independent databases and fused database are carried out to evaluate the performance of the proposed system. Our novel method achieves higher accuracy compared to previous literature, with the added benefit of low latency for inference.