Research And Application Of Facial Expression Recognition Method In Dual Branch Network

With the rapid development of deep learning methods,facial expression recognition methods have also achieved rapid development,but there are still many difficulties.For different people,even if they express the same type of expression,there will be certain differences.In real life,people express their expressions while being accompanied by words.These words will cause changes in mouth shape and thus bring about the difference in expressions caused by these situations will seriously affect the accuracy rate of recognition.At the same time,expression is a dynamic process.In the past,twodimensional convolutional neural networks were used to extract features from a single frame of images,ignoring the timing information in the process of expression changes,resulting in a waste of feature information,and also affecting the accuracy rate of expression recognition.In order to solve the above problems,the corresponding mitigation methods are proposed respectively in this thesis,and a dual-branch network facial expression recognition method is also proposed by combining these methods.The main work content of this thesis includes the following three parts:(1)Aiming at the problem of differences in facial expressions,a method based on input data is proposed in this thesis.This method discards the mouth region images with the largest differences in facial expressions to reduce interference features.Based on 3sets of comparative experiments,for data collected under laboratory conditions,this method can only improve the recognition accuracy of a few expression categories,but for expression data from real life,it can improve the recognition accuracy of 50% of the expression categories.(2)Aiming at the problem of the waste of time-series information in the process of expression change,the three-dimensional convolutional neural network is used in this thesis to extract the features of the expression image sequence.Under the condition of ensuring the ability of spatial feature extraction,the timing information implied in the sequence can also be extracted.At the same time,combined with the mitigation method of the difference problem within the expression category,a two-branch network structure is proposed in this thesis.The network uses two branches to deal with the problems mentioned above,respectively.Based on two sets of experiments,the network proposed in this thesis can achieve 98.71% and 92.52% expression recognition accuracy rates on the CK+ dataset and the Oulu CASIA dataset,respectively.(3)Based on the dual-branch network facial expression recognition method,a realtime facial expression recognition system is designed and implemented in this thesis.The system consists of two parts: a robot and a mobile phone APP,and uses ROS communication mechanism as the main method of communication.After the user completes the binding between the mobile phone APP and the robot,the real-time facial expression recognition results of the robot can be observed through the mobile phone APP.