Occluded Facial Expression Recognition Based On Deep Learning

In human’s daily social interaction,facial expressions often contain rich emotional state and intention information.Facial expression recognition technology can make computers understand human emotions and psychology.It has certain potential application value in many aspects such as human-computer interaction,fatigue driving monitoring,lie detection,and mental illness detection.In the real world,human faces are subject to unpredictable occlusions due to the complex and changing external environment,resulting in poor performance of expression recognition algorithms in traditional constrained environments.Therefore,occluded facial expression recognition in real-world environment is the focus of current research.For this,after a systematic and comprehensive research,this thesis analyzes the advantages and barriers of existing related algorithms.And it builds a multi-occlusion and multi-age facial expression dataset and studies new occluded facial expressions recognition algorithms.Besides,it verifies the superiority and robustness of the algorithms on many open-source large-scale facial expression datasets.The main work of this thesis is summarized as follows.First,the current technology related to emotion recognition based on convolutional neural networks are studied to explore their relevance with expression recognition technology.Next,the background and significance of facial expression recognition technology,related cutting-edge research at home and abroad and cross-application with other fields are summarized.Then,the limitations of the existing expression recognition research are analyzed,and the key scientific problems that need to be solved for facial expression recognition tasks are further summarized.Second,to address the problem of missing occluded facial expression datasets with considering the correlation between age and expressions,a multi-occlusion and multi-age facial expression dataset is constructed,containing a video set and corresponding image sets.All samples in the datasets are labeled as seven discrete expressions of angry,contempt,fear,happy,neutral,sad,surprise.And each expression sample is further labeled as three age levels of old,middle-aged and youth,children.Also,benchmark experimental results under various tasks are provided.Subsequently,a self-attention mechanism is introduced to solve the difficulty of focusing on unobstructed regions of occluded facial images,and a facial expression recognition algorithm with interrelated fusion is proposed.The algorithm constructs a new local enhancement preprocessing method,which enables it to obtain key discriminative features from three semantic dimensions:local region,context,and overall image.It achieves effective decoupling of unoccluded facial expression regions by reasonably grouping the face patches according to their criticality through analogue pooling units based on statistical indicators.Next,it further focuses on the reinforcement and complementary information between local and global face patches,to improve the accuracy of facial expression recognition effectively.Finally,the semi-supervised learning algorithms and theirs applications in computer vision are reviewed.To address the problem of difficulty in local regions feature extraction and representation of occluded facial expressions and to further make full use of available expression image data,a similar semi-supervised learning local region enhancement algorithm is proposed.It is built based on the CutMix local region data enhancement method that introduces a channel-spatial attention mechanism to focus on the key regions of expressions and incorporates the idea of semi-supervised learning to enhance the facial region recognition feature extraction ability.It realizes local feature-level enhancement,as well as improves data utilization and expression recognition performance.