Research Of Facial Expression Recognition Based On Facial Feature Point Analysis

As we know,expression can reflect emotion activities.Researches regarding expression recognition and emotion perception have become a hotspot in the field of human computer interaction.In recent years,the video-based facial expression recognition has received increasing attention due to its non-contact collection,convenience,and low cost.Currently,the research object of facial expression recognition can be divided into two categories:static image and dynamic image according to the way of information acquisition.Compared with static images features,the dynamic image feature parameters are more effective to reflect the motion correlation among different expressions because the change of emotion is a gradual process.Therefore,this thesis focused on the analysis of the variation of feature point position under different emotions using dynamic image sequence.On this basis,we developed a facial expression recognition algorithm based on Active Shape Model(ASM).Considering the involuntary facial deflection and different illumination,the paper proposed an improved Scale Invariant Feature Transformation(ISIFT)to enhance the robustness of expression recognition based on the traditional Scale Invariant Feature Transformation(SIFT)algorithm.The main contributions of this paper can be summarized as follows:(1)Investigated the current status of emotion recognition research and introduced the non-contact and contact emotion recognition method.The video-based and speech-based emotion recognition algorithm were presented in the part of non-contact methods.Meanwhile,we introduced physiological signal-based contact emotion recognition method.Additionally,the two common emotion research databases of JAFFE and CK+were introduced in detail.(2)Designed a face expression recognition algorithm based on ASM by extracting expression features of dynamic emotion sequences.To start with,the theoretical basis of ASM algorithm was introduced in detail.Then,we applied the ASM algorithm to locate facial feature points and selected 20 feature points from the significant area of facial expression change.Furthermore,the geometric features were extracted from the position information of the 20 feature points to indicate the relevance of the expression generation process.Finally,the extracted expression features were used to construct feature vectors for classification using Support Vector Machine(SVM).The experimental results showed that the ASM method had a good performance in facial emotion expression.(3)Researched a facial expression recognition algorithm based on ISIFT.Viola-Jones algorithm and Constraint Local Model(CLM)algorithm were used to detect face and locate feature points,respectively.Then,we selected 42 feature points from the most active area of facial expression to extract SIFT feature.Principal Component Analysis(PCA)algorithm was used to reduce the dimension of extracted features,and the SVM was used as the classifier.On the frontal facial database,the recognition ratio of facial expression using the ISIFT algorithm obtained 98.59%.In addition,the experimental results under different facial poses showed that the ISIFT algorithm could detect the human face within an angle range between 0 and 10 degrees effectively.The average expression recognition ratio was 95.18%and 94.34%when the face poses deflected to the left or right,respectively.At the same time,four different lighting environments(no lights,left lights,right lights,two lights)were designed,and the corresponding expression recognition ratios were 94.76%,95.59%,95.31%,and 95.36%.The results showed that the ISIFT algorithm was robust in different illumination environments for facial expression recognition.(4)According to the researched ISIFT facial expression recognition algoritlhm,a video-based facial expression recognition system was designed.The system was divided into offline and online module.For the offline module,a single frame image could be selected from the database to determine the emotional state.For the online module,the facial expression of the subject could be recognized in real time.Both modules had the functions of face detection,feature point location and expression recognition.