Research On Dynamic Expression Recognition Based On RealSense

Facial expression is one of the important ways of human emotional communication.Expression recognition has become a key issue in the field of emotional computing in the field of education.The development of portable 3D sensors and mobile devices has also brought more possibilities for new human-computer interaction methods,so that the application of facial expression recognition can more naturally integrate into smart teaching.This paper summarizes the research work on facial expression recognition by scholars and finds that the existing researches are mostly based on the static facial expression recognition of ordinary RGB sensors that acquire planar 2D images.Compared to dynamic expressions,static expressions lack continuous information.However,expression recognition based on 2D images is susceptible to illumination,and shooting angle.The use of 2D images to describe 3D faces is bound to cause some important information loss or distortion,thus affecting the accuracy of expression recognition;some use of 3D sensor recognition method,the cost of the sensor is expensive,the feature extraction process is more cumbersome,is not conducive to the application of expression recognition.In order to overcome the technical deficiencies above,the RealSense 3D sensor is used in this paper to obtain facial information.After acquiring facial landmarks,the dynamic geometric features of the face are extracted and transformed into semantic feature descriptors.The SVM classification method is combined with a classification method based on local feature description labels to realize dynamic three-dimensional expression recognition.The main work is as follows:(1)Create RealSense 3D Dynamic Emotion Database.In view of the fact that there is no database of facial expressions based on RealSense in the world,the research team has organized and recorded the RealSense expression database that contains coordinate information of facial RGB-D,head posture,and 78 facial landmark points.The database contains six basic expression categories(happy,sadness,surprise,anger,fear,disgust).(2)For face data containing RGB-D information,a normalization method based on three-dimensional space is used.By analyzing the characteristics of each expression,facial expression features are built on the face region,eye,eyebrow,mouth,and nose.The geometric features of three-dimensional dynamic expressions are extracted based on the facial landmark points,including distance features and angle features.(3)An expression classification method based on SVM and regional feature description label fusion is proposed.During the fusion process of the classification method,the SVM classifier was trained using 3D geometric features to perform initial classification.By outputting the posterior probability of the SVM,the fuzzy samples in the SVM classification process were selected,and the secondary classification was performed using the classification method of the regional feature description label.The classification method of the regional feature description label transforms the geometric feature from the feature space to the semantic space by constructing a tree feature description tagger,completes the tagging of the feature description label,and obtains the contribution degree of the feature description label to the expression class through iterative training.Construct feature description-category matrices and use BOW matching methods to determine expression categories.The fusion of classification methods effectively reduces the problem of poor generalization of classification methods due to sample differences in expression recognition.Combining the three-dimensional dynamic expression recognition algorithm proposed in this paper,using the VS2015 integrated development environment,combined with RealSense3D sensor,built a set of expression recognition system under the Windows platform.The RealSense expression library and CK+expression library are used to verify the effectiveness of the classification method.