Research On Facial Expression Recognition Based On Deep Convolution Neural Network

Facial expression is the expression of human emotions,and facial expression recognition has wide application in human-computer interaction,machine vision,surveillance,lie detection,and health care.Automatic expression recognition with face image/video has become one of the important research topics in the fields of emotion computing,image processing and pattern recognition.It has been highly concerned by researchers in the past few decades.Especially the great success of deep convolution neural network in the field of image recognition has aroused a new upsurge of expression recognition research based on deep learning.However,when facial expression recognition methods are applied,there are still many challenging problems to be solved.In this thesis,novel deep convolution neural network based facial expression methods are studied,aiming at the problems that need to be solved in the application of facial expression recognition,such as spontaneous facial expression recognition,the influence of facial features on facial expression recognition,and the distinction of expression intensity.The main innovative achievements are as follows:1)Spontaneous expression recognition based on deep transfer learning.Most of the existing expression recognition methods are based on artificial expression image database for classifier training,so it is difficult to recognize the spontaneous facial expression effectively in the actual application environment.In order to solve this problem,this thesis first proposes a method of constructing spontaneous expression data set with spam image filtering model and Internet images.Considering that the amount of training data is not enough when the data set is used for deep learning,we further use deep transfer learning method for spontaneous expression recognition.The experimental results on the CK+ standard dataset and the self-built expression dataset show that the proposed method improves the expression recognition accuracy,and the combined network of VGG-face and Re Net works best.2)Personalized facial expression recognition based on mixed deep network and multi task learning.The existing facial expression recognition methods rarely take into account the differences of different people's facial expressions,so it is difficult to achieve personalized facial expression recognition effectively.In order to solve this problem,this thesis transforms the facial expression recognition problem of different people into a large-scale multi task classification problem of different people and different expressions,and proposes a personalized facial expression recognition method based on multi task learning of mixed deep convolution neural network.Firstly,the multiple basic networks are fused into a mixed deep network with more dimensional output to solve the large-scale classification problem;secondly,the task group automatically generates and optimizes the overlapping percentage of tasks among groups through hierarchical clustering,so that the expression images with high similarity can be trained together;then,all the basic deep convolutional neural networks are learned together in an end-to-end way to improve improve the recognition of similar facial expression images by mixed deep network.The experimental results show that the proposed method can effectively carry out personalized facial expression recognition for a large number of people.3)A fine-grained expression recognition method based on deep multi task hierarchical classification.The existing expression recognition methods only divide expression into basic expression,while seldom consider the influence of different face types on expression recognition and the different intensity of expression.Therefore,it is difficult to meet the needs of accurate discrimination of expression intensity in practical application.To solve this problem,this thesis proposes a fine-grained expression recognition method based on multi task hierarchical deep network.Firstly,in order to reduce the influence of face shape on the result of intensity classification,an expression recognition method based on Deep Convolutional Neural Networks(DCNN)multi task hierarchical classification is proposed,which uses face shape attribute to learn more recognizable deep expression features,so as to reduce the influence of face shape on expression recognition;secondly,an expression intensity classification method based on the probability combination of multi granularity classification is proposed.Through the probability combination of rough classification and fine classification results of expression,the differentiation between adjacent expression intensity levels is improved.The experimental results show that the proposed method effectively improves the facial expression recognition accuracy.The experimental results on the UNBC-Mc Master shoulder pain dataset show that compared with the existing deep network method,the proposed method improves the accuracy of pain classification.