Research On Facial Expression Recognition Based On Deep Learning

With the rapid development of artificial intelligence,society has gradually shifted from intelligence to wisdom,and facial expression recognition is one of the important research directions.In traditional methods,researchers mainly set the selected features in advance by themselves,and then extract them in the process of experiment,which is complicated and it is easy to lose deep semantic information,which affects the final expression recognition effect.In response to such problems,this thesis designs a deep neural network structure for feature extraction,so as to obtain deeper facial expression feature information.The main research work and results are summarized as follows:1.Aiming at the problem that the current facial expression recognition methods are insufficient generalization ability in terms of illumination,noise,etc.,and the public data set has a small magnitude,this thesis proposes a facial expression recognition method based on convolutional neural network,histogram of oriented gradient and facial landmark detection.First,this method uses multi-task convolutional neural network in image prreprocessing to detect faces of different scales of input images and obtain landmark position information of the face;Then,the geometric structural features of the facial expression image are calculated according to the location information of the facial landmark,and HOG features are composed of the direction gradient histogram of the local region of facial expression image,which is further fused with the geometric structure feature of facial landmarks by feature fusion method to form a new feature vector LM?HOG;Finally,the LM?HOG and the global features extracted by CNN are fused again and input into the classifier to complete expression recognition.The experimental results show that the fusion network using multi-channel feature extraction can effectively complement for the defect of single feature,and achieve great recognition effect in the task of facial expression recognition.2.Aiming at the problem that the gradient value returned by the back-propagation of the neural network disappears when the layer is too deep,which makes the network difficult to train,this thesis proposes a facial expression recognition method based on the deep residual network model,which modifies the original convolutional pooling structure into a residual unit in the feature extraction stage,thereby improving the network model building ability and image feature learning ability.Finally,the output features of the deep residual network are used as the input of the classifier,so that facial expression recognition can be completed.The experimental results show that the facial expression recognition method based on the residual network avoids the gradient explosion and dispersion problems during deep network training,and achieves a good recognition effect in the Res Net50?v2 network structure.3.Aiming at the problem that the convolutional neural network achieves "position invariance" during pooling operation,but at the same time it inevitably leads to the loss of some details of the image,this thesis proposes a facial expression recognition method based on capsule network,which stores the feature information of facial expressions through vector entities and saves the pose information between entities,so that the capsule network can further achieve "identity" on the basis of the "invariance" of convolutional neural network.The experimental results show that the facial expression recognition method based on the single-layer capsule network can recognize the facial expression better by combining the entity information and pose information of the current face image,and the accuracy and generalization ability are significantly improved.In order to improve the feature extraction ability of the capsule network,a double-layer capsule network model is proposed for expression recognition by deepening the depth of the capsule network.In this method,inorder to avoid the huge time cost caused by multiple routing layers,an improved feature extraction network is used,and proposes an operation based on deconvolution to improve the reconstruction network.Furthermore,the pruning strategy dynamic routing algorithm is further used to solves the poor learning problem of the middle layer caused by the fully connected capsule layer.The experimental results show that,compared with the single-layer capsule model,the double-layer capsule network can achieve better feature extraction capabilities and a higher recognition accuracy rate.Figure[60] Table[23] Reference[88]...