Multi-feature Fusion For Emotion Recognition Based On Deep Learning

Emotion is an important carrier of communication in which people express their feelings.Effective interpretation of emotion can help us to understand other people's intentions and reactions.With the gradual development of deep learning and pattern recognition technology,emotion recognition technology continues to make breakthroughs and is widely applied in real life.Emotion recognition is very useful in psychology,public safety,human-computer interaction and other fields.At present,most of the researches focus on emotion recognition of facial expressions,which are seldom involved in gesture expression.With the commercialization of emotion recognition,the algorithm itself is more and more demanding for the robustness of complex environment and light.Based on the above status quo,this paper proposes an improved method for the common recognition of facial expressions and gesture expressions based on deep learning,which is accompanied by experimental verification.The main research work is as follows:1.The database of face and gestures expression is established.This paper expands it according to the shortage of existing database,and implements data enhancement by random clipping,rotation and so on.In addition,this paper also generates optical and differential expression images with time series information.2.This paper combines Googlenet and SE module to build a static expression recognition structure based on convolution neural network.The experiment shows that the structure has good background and illumination adaptability.At the same time,this paper verifies the effectiveness of data enhancement in solving the over fitting problem.3.In view of the shortcomings of static expression recognition,this paper builds dynamic expression recognition model based on LSTM.Through migration learning,the model can achieve expression recognition in a shorter training time.4.In this paper,a LSTM model based on two-channel input feature fusion is proposed,and the convolution kernel size matching problem in convolutional neural networks is solved by using pyramid pooling layer.This article also verifies this conclusion by a large number of comparative experiments.Through theoretical analysis and experiment,we know that TSLCN proposed in this paper is suitable for most kinds of complex environments,has high robustness and recognition rate,and is of great significance for the further study of expression recognition.