Research On The Technology Of Pose-invariant Facial Expression Recognition

Facial expression recognition(FER) in the wild is one of the key objectives of human–computer interaction(HCI). It is, however, very hard to realize bescause the facial images are often captured form various poses as a result of the head movements. Thus if we want to apply the FER system to the real world, the problem about the changing poses is yet to be resolved. Based on how they deal with variation in head-poses and expressions in facial images, the systems can be divided into three categories: 1) methods that learned separately for specific poses, 2) a single classifier learned for all views and ignored the influence caused by different poses, and 3) methods that performed pose normalization first before using for pose-invariant FER. For the first method, it need separate training and parameter tuning for each pose. For the second method, the final results are easily affected by the changging poses. The results of the third classifier are usually dependent on the accuracy of the landmark location, because it needs to learn the mapping between different poses through the landmarks. Accurate landmark detection, however, in arbitrary images continues to be a major challenge. To solve these problems, we present a novel cascaded multi-level transformed dirichlet process(cml-TDP) model and a spatially coherent feature learning method(Spatial-PFER) for pose-invariant facial expression recognition. The major contributions of this thesis are:(1) A multi-level transformed dirichlet process for parts segmentation and facial expression recognition. To avoid separate training and parameter tuning for each pose, we present a novel cascaded multi-level TDP model for key parts segmentation, and pose-invariant FER. Pose is explicitly introduced in cml-TDP to learn a relationship among different views, thus the key parts segmentation and FER can be finished in a unified TDP model. A facial image in our model is described by the segmented parts including the positions and corresponding appearance features. The geometric constraints among different facial parts are implicitly encoded in our model and integrated with the local features to facilitate the pose-invariant FER. Thus, cml-TDP can obtain high recognition rate even with large pose variation. Experiments on two benchmark facial expression databases(BU-3DFE and RAFD) show the superior performance of our system.(2) Spatially coherent feature learning for pose-invariant facial expression recognition. To avoid complex landmark location and multi-ple modle training with the increase of poses, we present a novel spatially coherent feature learning method for fast pose-invariant FER. We first carry on a face frontalization process through a 3D pose normalization technique, which could synthesize frontal face from the image levels. Then KR-AE carries on the unsupervised feature learning method auto-encoder based on many key regions, which could effectively improve the training efficiency and FER accuracy. Finally, we introduce a linkage structure over the learning-based features and the corresponding geometry information of each key region to encode the dependencies of each region, which could effectively increase the robustness of the model. We validate the proposed KR-AE model on the benchmark facial expression databases(BU-3DFE) through many aspects. The results show that our model outperforms the state-of-the-art methods for pose-invariant facial expression recognition.(3) Implementation a facial expression recognition GUI. A prototype system for pose-invariant facial expression was realized by using MATLAB and C++. The functions include face detection, key parts segmentation, facial expression recognition, et al.