Study On The Algorithms On Facial Expression Recognition

Facial expression recognition and classification play a key role in the research field ofcomputer vision and pattern recognition. From the1980’s, computer based facialexpression recognition has gain more and more attention of researchers all around the world.At the meantime, as a branch of Artificial Intelligence and affect computing, facialexpression recognition has tremendous research value and application prospect inHuman-Computer Interaction and Intensive Care Unit night watching. The early researchon facial expression recognition mostly focuses on data format of2dimension pictures andpicture sequences, namely2dimension videos. But the2dimension pictures are justprojection of the3D object to the2D plan. As in the projection process, there must be someloss of the feature and shape context, and it will be impact hugely by the gesture of humanbeing and illumination. So, in the past ten years,3D points cloud (3D mesh model) basedfacial expression recognition get into the center of this research field and gradually becamethe emphasis of this research topic.In this dissertation, we first introduce two fully automatic facial expressionclassification algorithm procedures on the3D points cloud expressions data set. Thenpropose three different algorithms to extract facial expression feature vectors which couldlead to precise expression classification rate compared with state-of-art algorithms. Finally,we will introduce a2D picture facial expression data set, which collect facial expressionpictures from the website by key words searching from the widely used commercial searchsuch as Goolge and Bing. The creative work of this dissertation is as follows:1) Introduced a procedure to automatically classify six basic expressions within the3Ddomain. Firstly, we got fiducial points from3D mesh models and the depth images andtexture images automatically. Then projected the fiducial points found in the depth imagesand texture images back into the3D human face points cloud to get the full range offiducial points in every face mesh model. Then we calculated a bunch of Euclidian distancebetween those fiducial points which designed previously as feature vector to feed classifierand got a wonderfully correct rate on six basic expression classifications. The mostsuperiority of the algorithm introduced is it could handle the expression feature vectorextraction fully automatically.2) A fully automatic facial expression recognition system on3D expression meshmodels was proposed. The features extracted from3D expression mesh model were a bunch of radial facial curves to represent the spatial deformation of the geometry featureson human face. Each facial curve was a surface line on the3D face mesh model, begunfrom the nose tip and ended at the boundary of the previously trimmed3D face points cloud.Then Euclid Distance was employed to calculate the difference between facial curvesextracted from the neutral face and each face with different expressions of one person asfeature. Finally we proposed an automatic feature vector select algorithm to pick theexpression feature vectors with the biggest distinction rate of different expressions.3) A new algorithm taking the spatial context of local features into account by utilizingcontextualized histograms was proposed to handle the facial expression recognition task.The contextualized histograms were extracted from two widely used macro-texture patterndescriptors which were local binary pattern (LBP) and weber local descriptor (WLD). Theface images could be represented pretty well by these macro-texture pattern descriptors.The histogram vectors generated by these descriptors contain rich discriminativeinformation for different expressions. So, in order to measure this discriminativeinformation, we used Histogram Contextualization algorithm to handle this task. Theexperiment results showed that, this method could improve the recognition rate of theoriginal feature vector remarkably.4) Proposed a system to handle the automatic facial expression recognition task. Wefirst extended the newly proposed2D texture analysis algorithm weber local descriptor(WLD) into spatial-temporal domain by extracting WLD features from each adjacent frameand two orthogonal planes composed of correspondence columns and rows from eachframe. Then contextualized histograms were extracted to represent the spatial context ofWLD feature vector. And the expression recognition rate of proposed procedure wasstate-of-art.5) Proposed a new method to use the extended Shape Context descriptor to representthe coded LBP image and got the feature vector for expression classification along with thisprocedure. First, we coded a facial image with LBP, when the whole image was coded, weuniformly and densely sample the LBP coded image with sample points. At every samplepoints, using the extend rotation invariant Shape Context proposed to extract feature vector.The extend rotation invariant Shape Context means using the main gradient direction as theorigin of the polar coordinate system. We got a sub histogram from every sector within thenet-like polar coordinate patch of the Shape Context descriptor. Then concatenate all thesesub histograms into feature vector to train the classifier. 6) The construction of facial expression data set is important for the expressionclassification task because all the algorithms developed to handle this task need trainingand testing facial data. But the existing facial expression data sets have some basic defects,such as lack of human object for whom the facial expression picture or video were taken,the facial images and expression video in every data set are rare, and all the data collectedare pictured from the extremely posed human objects, it’s far from the real expression whathuman wear on their faces every day. So, we introduced a2dimensional facial expressionimage data set constructed by the picture collected from commercial search engine. Theframe has two steps: collect the original facial pictures with expression on them by keyword searching from commercial search engine; then refine the original data set by activelearning algorithm to get the final data set.