Research And Implementation Of Face Expression Recognition In The Wild Based On Convolutional Neural Network

Human Facial expressions play an important role in daily life,providing significant clues concerning human beings' intentions and emotions.How to make the computer correctly recognize and understand facial expression information is a significant and challenging work.With the development of deep learning,expression recognition methods based on deep learning have made significant advances and being a new research hotspot.Most facial expression features are concentrated in the local key areas of the face,such as eyes,eyebrows,mouth and its surrounding areas.People can easily recognize emotions in cases of complex conditions through discriminative facial local regions,such as the eyes,eyebrows,mouth,and the other surrounding areas,instead of the whole face.Therefore,how to enable the deep neural network to extract more discriminative facial features from the face images in the wild,and improve the overall recognition accuracy of the facial recognition network,is the focus of facial recognition research.Most of the existing facial expression recognition methods focus on the key points of the face,and make multiple face regions,then send them into the network,which try to highlight crucial local facial regions by leveraging attention weights to adjust the importance of facial sub-region adaptively.Although they have achieved promising performance,they also require complex preprocessing and training,and thus they introduce an abundance of calculation with facial landmarks detection and cropped operations.In fact,the face in the wild meets complex background,which is often affected by different factors,such as illumination,occlusions and head posture,etc.Of course,it is of great importance and practical significance to facial expression recognition in the wild.To this end,we mainly aim at achieving the facial expression recognition in the wild and designing a real-time expression recognition system.Specifically,the research work of this paper are as follows:First of all,we propose a destruction and reconstruction learning convolutional neural network named ADC-Net for expression recognition.The ADC-Net learns more discriminative expression features from crucial local facial regions by shuffling the spatial layout of original facial images and reconstructing the whole image.Specifically,the input face image is first divided into local sub-regions of the same size,and these local sub-regions are randomly scrambled and reordered within a certain range to get a new damage image,which increases the difficulty of recognition.Then the original image and the damaged image are sent into the trunk network for feature extraction,and a channel attention module is used to enhance the effective features and suppress the invalid features.At the same time,in the reconstruction process,the regional alignment network is added to try to make the network restore the original spatial layout of the local sub-regions in the original image,and establish the semantic association model among the local sub-regions.The experimental results show that the proposed ADC-Net network in this paper can achieve better performance in two facial expression datasets RAF-DB and FERPlus,with the recognition accuracy of 88.46% and 88.9%,respectively.Secondly,we adopt the gradient-weighted class activation mapping,called Grad-CAM to study the proposed ADC-Net for "visual interpretation",and generate the corresponding attention heatmaps of the input image,in order to reflect the influence the result of the classification.It can be clearly seen from the generated heat map that ADC-Net pays attention to the facial expression features of key local regions of the input face image,which proves that the ADC-Net can extract more discriminative facial expression features from these key local regions.In this paper,the robustness of ADC-Net also can tackle the occlusion and pose problems by using the selected image sets with occlusion and pose problems in the RAF-DB dataset.The experimental results show that ADC-Net has good robustness to deal with such problems.Finally,we design a real-time face expression recognition system based on Py Qt and the ADC-Net expression recognition network proposed in this paper.The facial expression recognition system has three modes: from local image,from video,and from real-time camera,which further verifies the effectiveness and real-time performance of the algorithm proposed in this paper.