| Expression recognition technology can analyze human emotions,provide users’ emotional feedback,detect fatigue driving and work,etc.,and has broad application prospects in many fields.At present,the deep learning method has achieved very good results for the research on facial image expression recognition without occlusion of the frontal face.However,in natural scenes,hair,hands,arms,food,etc.may cause occlusion,these objects may block any part of the face,and the accuracy and robustness of expression recognition will drop sharply.Expression recognition under occlusion is still a huge challenge due to the different locations,sizes,and shapes of occluded regions.Therefore,how to design an expression recognition algorithm that can solve the problem of partial occlusion is a topic of research significance.In view of the above problems,this thesis decomposes the problem into two sub-problems:repair of occluded face images and repaired expression recognition.The specific work is as follows:(1)A fast face inpainting algorithm is proposed to repair the occluded part of the face.The algorithm uses the symmetrically Speed-Up Robust Feature(SURF)to quickly detect the central axis of the face,and then combines the facial key point detection technology to locate the face occlusion area and establish a coordinate system.Next,an unsupervised face restoration module based on mirror transformation is designed to restore the occluded area according to the established coordinate system and key point coordinates,and quickly complete the face restoration task without changing the information of the non-occluded area.The impact of partial occlusion improves the accuracy of subsequent expression recognition algorithms.(2)An expression recognition network combining global multi-scale and local heterogeneous weighted features is proposed.The network consists of a conditional independence-based feature extraction module and two network branches.In the first branch,a multi-scale module is designed to learn global multi-scale features of face images,while the other branch is designed with heterogeneous.The weighting module is used to learn the local features of the face image.Finally,the global features and local features go through the global average pooling layer and the fully connected layer,respectively,and perform decision-level fusion on the expression recognition classification results.The feature extraction module in our method can remove noise and features irrelevant to the expression task,the global multi-scale feature increases the diversity and robustness of features,and reduces the sensitivity of deep wise convolution to occlusion,while the heterogeneous weighting mechanism guides the network focuses on face parts without occlusion,and the extracted local salient features increase the robustness of the network.(3)Due to the lack of expression recognition data under sufficient occlusion,this thesis increases the amount of data by adding random graffiti to three datasets of Celeb A,CK+ and fer2013,and uses the three datasets with graffiti to test face inpainting algorithms and occlusions performance of facial expression recognition algorithms.The experiment uses two objective evaluation indexes PSNR and SSIM to evaluate the effect of inpainting images.The experimental results show that the method in this thesis not only has fast data processing speed,but also has good image inpainting effect in the face inpainting task.In addition,in the expression recognition task,the proposed method has achieved excellent results on multiple datasets,especially on the CK+ dataset with graffiti,which achieved an expression recognition classification accuracy of 88.3%. |
PDF Full Text Request