Research On Facial Expression Recognition In The Wild Based On Convolutional Neural Network

Facial Expression Recognition(FER)in the wild has attracted widespread attention in the field of computer vision recent years,as it holds promise to an abundance of applications,such as human-computer interaction,expression simulation,and driving fatigue monitoring.At the meantime,with the development of deep learning,the recognition accuracy of FER systems based on Convolutional Neural Network(CNN)has been rapidly improved.Although the performance of existed FER systems has reached an acceptable level,the majority of them are built on images captured in the constrained environment.The expression recognition in the unconstrained environment is still a challenging task.To fill the gap between the FER accuracy on the collecting faces and un-controlled faces,researchers have made exploration efforts,such as collecting the facial expression database,proposing innovative algorithms,as well as improving the structure of CNN.Despite the efforts above,effective extraction of the expression-related features in the unconstrained conditions remains a challenging task.FER in the wild is a challenging task because it may encounter many inevitable issues that nonlinear coherent to expression in spatial.Therefore,the extraction of expression-features is the hardest part of expression recognition.This paper was conducted research work on the extraction of the features related to expression in the wild.The contributions of the work are as follows:(1)A multi-channel convolutional neural network model with decision mechanism is proposed.The strategies are used to solve multiple problems encountered in facial expression recognition in the wild.In the pre-processing phrase,an illumination augmentation strategy is applied to process the facial images to improve the robustness of illumination changes.Then the assisted channels with Decision-Net are designed to extract the features of the regions with less occlusion.Besides,Expression-related features of the whole face are extracted by a pretraining neural network that serves as the main channel of the model.Finally,a dynamic weighing strategy is proposed to fuse the outputs of multiple channels.The experimental results on three public databases demonstrated that proposed models are competitive and representative in the field of FER in the wild research.(2)A double-channel convolutional neural network with occlusion perception is proposed to recognize facial expression from partially occluded faces.Region Decision Unit is designed and integrated into the VGG16 network to form a neural network with occlusion perception,which aims to extract expression-related features of the areas with less occlusion.In the convolutional layer training phase,the transfer learning algorithm is adopted to pre-train the parameters of the convolutional layer,which means to alleviate the problem of the over-fitting.At the meantime,the expression-related features of the whole facial image are extracted by the modified residual network.Finally,the outputs of both channels are fused in a weighted manner.The experimental results on three public databases demonstrate that proposed models are competitive and representative in the field of FER in the wild research.Compared with traditional approaches,the proposed model effectively improves the recognition accuracy.