Occluded Facial Expression Recognition Based On Contour

As the most intuitive form of human emotion expression,facial expressions have always attracted the attention of scholars around the world.It has been an active research field of deep learning.Currently,the expression identification comprehensively considers the morphology of mouths,eyes and other parts.Since facial data for experiment is tidy and clean,and the key parts are clear,it is easier to implement.So far,many models have achieved high accuracy.However,most faces in real scenes have occluders.For example,people have to wear masks according to the epidemic prevention requirements.Facial expression recognition for faces with occluders is still a hard obstacle in research because that occluders hide the key part information,which make it impossible for a model to judge.Moreover,the position of the obstruction is random,therefore,the disappearance of features is also random.Consequently,it is impossible to pass fixed features to a model for training,as well as the type of features received is different each time,which decrease the accuracy of model.In view of the problem that most of the face images collected in engineering applications have occluded objects,this thesis first repairs the occluded image,and repairs the occluded parts as much as possible,and then recognizes it.The focus of this thesis is to repair occluded images.If the occlusion can be repaired better,the recognition accuracy will naturally be higher.The main research work of this thesis is as follows:1.Repair the occluded image in two stages: the first stage repairs the contour of the occluded image.Compared with the original image,the outline highlights the morphological characteristics of the image.By comparing the contour map of the occluded image with the non-occluded image,the neural network can easily capture the contour information of the occluded object to repair;the second stage is to merge the restored contour image with the original image of the occluded object.At this time,the repaired contour map is used as a priori knowledge,which can guide the neural network to lock the position of the occluder,and can provide the geometric characteristics of the part to be repaired to help the network repair.2.Use the ResNet network as the classifier to classify and train the restored images separately,and use the feature maps in the process of classifying the restored images as labels for the occlusion image classifier to learn.Achieved 93.4% accuracy3.Applying neural network to recognize facial expressions of students in classroom teaching with occluders,and design and implement a recognition system for student facial expressions to assist teachers in teaching.4.In the process of constructing the contour restoration network,it is found that adding a fully connected layer can eliminate the contour of the occluded object and repair the contour of the occluded part.When the fully connected layer is not added,the network can only eliminate the contour of the occluder,but cannot repair the contour of the occluded part.It is guessed that it is caused by the difference in the feature calculation method between the fully connected layer and the convolutional layer.And make a comparative experiment to verify.