Facial Expression Classification In-the-Wild Based On Deep Learning

Facial expressions have an important role in conveying emotional information in communication.Facial expression recognition(FER)has important applications in the fields of human-computer interaction,distance education,and safe driving of vehicles.At present,the research of FER has shifted from laboratory scences under controlled environment to natural wild scenes,from traditional methods based on manual and shallow features to deep learning methods.At present,there are two main problems in FER: one is that there are fewer facial expression training samples,lower resolution,inaccurate category labels,and unbalanced categories,which increase the difficulty of training;the other is that feature extraction methods adopted by existing algorithms have limitations,and the extracted features are mostly shallow features.In addition,facial images in natural scenes are highly susceptible to environmental factors such as facial poses,background illumination,and partial occlusion.Therefore,feature extraction methods based on traditional manual and shallow features cannot meet the needs of FER in natural scenes.In view of the problem of insufficient feature extraction,we propose a deep feature extraction network based on Inception-Res Net.The network combines the benefits of multi-scale receptive field in Inception network and residual learning to efficiently extract deep features of facial expressions.This method solves the problem that traditional manual features and shallow features cannot meet the requirements of FER task under natural scenes.In view of the insufficiency of the number of expression samples,it is impossible to train deep neural networks from scratch.We draw on the method of network-based feature transfer.First,we initialize the network with the pre-trained weights on Image Net classification task,and then retrain the entire network with the expression samples.This method has achieved good recognition results on several expression data sets such as CIFE.Based on the depth feature extraction network,we design a spatial attention module that can adaptively learn the weight distribution of expression features in space.This module solves the problem of insufficient extraction of local detail features in existing facial expression feature extraction work.It plays a role in guiding the weight distribution in the whole feature extraction network.On the one hand,it strengthens the weight of the region containing facial features in facial expressions,and on the other hand reduces the negative impact of background and irrelevant regions on recognition.The visualization of the spatial attention layer visually demonstrates how it adaptively affects the distribution of weights across face space.Experimental results on CIFE and other datasets show that our proposed model has good recognition performance and surpasses existing methods.The work of this thesis shows that the FER based on spatial attention and Inception-Res Net deep feature extraction network has important guidance and deep feature representation for expression feature extraction on small sample datasets.The spatial attention module we designed can guide the network to adaptively focus on the expression-related regions on face.At the same time,based on the Inception-Res Net deep feature extraction network,deep features of facial expressions can be extracted from the multi-scale receptive field,and expression recognition results can be improved.