Research On Expression Recognition Based On Dual-dimensional And Multi-features

Face expression recognition(FER)has been a hot research topic in the field of pattern recognition and artificial intelligence.It has broad application prospects in many fields such as human-computer interaction,video surveillance,safe driving,clinical medicine and so on.So,how to identify the face expression efficiently and accurately is of great significance.At present,the method of FER usually focuses on extracting features from 2D face images and analyzing the feature of local facial texture and contours to recognize facial expressions.Given the complexity and subtlety of facial expressions,distinguishing facial expression accurately only using 2D features extracted from 2D face images is difficult.So the recognition effect decreases drastically when processing non-database images or face poses and ambient light changes.Traditional 2D facial expression recognition methods are easily influenced by various factors,such as posture and illumination,and cannot effectively recognize a few confusing expressions.In this study,a method that combines 2D pixel features(2D-PF)and 3D feature point features(3D-FPF)based on Kinect sensor is proposed to achieve robust real-time FER and thus overcome the above-mentioned disadvantages of previous methods.The main research contents are as follows.The face image is segmented by the enclosing rectangle around the area of the eyebrows and mouth.As a result,the segmented face image does not contain the background part and the block of the forehead and chin;other irrelevant areas that do not reflect the expression changes are excluded as well.The classic LBP,Gabor,and HOG operators are used to extract2D-PF from the segmented face images.The computation of LBP,Gabor,and HOG feature extraction is generally relatively complex,thereby hindering the real-time operation of the algorithm.Accordingly,proper adjustments on the process of extracting LBP,HOG,and Gabor features are performed to reduce the computation cost.The eigenvectors are also reduced dimensionally to ensure the real-time performance of the algorithm.However,2D-PF presents difficulty in describing the feature changes in facial expression and is sensitive to various extraneous factors.Thus,three types of 3D features of angle,distance,and normal vector are proposed to describe the deformation of face in detail and thus improve the recognition effects on confusing expressions.After a detailed analysis of the changes in facial expressions,the 3D features of angular,distance,and plane normal vectors between the connection line of different feature points in eyebrows and mouth area are selected as feature vectors in describing the changes in facial expression.However,the small number of feature points in eyebrows and mouth area and the low precision of 3D data acquired with Kinect result in poor recognition effects.Accordingly,2D-PF and 3D-FPF are integrated to complete the recognition task and thus ensure the balance between the accuracy of expression recognition and real-time performance.The proposed expression recognition method can improve the expression feature describing capability of the algorithm by combining 2D-PF and 3D-FPF.It can effectively reduce the interference between confusing expressions and enhance the robustness of the algorithm by use of the weighting average of random forest classification.The effect of the algorithm is verified by recognizing 9 different expressions including calmness,smile,laugh,surprise,fear,anger,sadness,meditation and disgust on the 3D expression database called Face3 D that contains 9 types of facial expressions of 10 individuals,with a total of 9,000 sets of images and feature point data.Experimental results show that the combination of 2D-PF and 3D-FPF is conducive to discriminating facial expressions and the average recognition rate of 9 expressions on the Face3 D database is 84.7%.The recognition rate can reach more than80% for a few confusing expressions,such as anger,sadness,and fear,and the real-time performance can realize 10 to 15 frames per second owing to the high-speed data acquisition with Kinect.The proposed expression recognition method can improve the expression feature describing capability of the algorithm by combining 2D-PF and 3D-FPF and can effectively reduce the interference between confusing expressions and enhance the robustness of the algorithm by use of the weighting average of random forest classification.The proposed method is more beneficial to the recognition of facial expression than ordinary 2D or 3D features and can guarantee insignificant decrease in real-time performance.