Research On The Spontaneous Facial Expression Recognition Based On Facial Landmarks

At the present,with the focus of research for intelligent machine,people increasingly hope that machines could assist and delight our lives.As an essential part of intelligent machine,facial expression recognition has important research significance and practical value.Facial expression recognition algorithm detects expressions based on the face images in the real environment which are captured by camera device.Machine makes further feedback through facial expression returned by the algorithm,and realizes the intelligent interaction with human.The performance of the most traditional facial expression recognition algorithms which are based on the face samples under experimental conditions will be limited when applied the face samples in the real environment.Face samples in the real environment have the following characteristics: spontaneous head poses,significant identity biases,illumination variations and complex backgrounds.Therefore,we should propose one recognition framework which is robust to above factors during the recognition of spontaneous facial expressions.In our work,we could recognize the spontaneous facial expressions in the real environment by means of the extraction of the discriminative features.The extracted discriminative features include two kinds of features: pose-and identity bias-robust(PIR)geometric features and multi-scale uniform local binary pattern(M-ULBP)appearance features.Regard to the extraction of geometric features,we calculate the weight vector of expression blendshapes of input images.The 3D vertices containing expression information could be achieved by the combinations of the weight vector of expression blendshapes,expression blendshapes and reference 3D vertices.The normalized landmarks which are unrelated to head poses and identity bias will be produced by the orthogonal projection transformation combined with the 3D vertices and reference projection parameters.The PIR features could be obtained by the calculations of distances and angles between the normalized landmarks.At the stage of the extraction of appearance features,we scale the input image into three resolutions to achieve the face image pyramid.For each scale,we segment the face image according to the selected facial landmarks to obtain the sub-regions,and ULBP features are applied to these sub-regions.The appearance features of input images could be achieved by concatenating the ULBP features from each scale.Finally,through experimental verification,the performance of the extracted features is superior to the same kinds of features.