Facial Expression Recognition Based On Optical Flow And Multiple HSMMs In The Presence Of Occlusion

In recent years, the reasons for renewed interest in facial expression recognition are multiple, but mainly due to people have more interest about human computer interaction (HCI). Facial expression recognition is to analyze and detect the special expression state from given expression images or video frames and then to ascertain the subject's specific inborn emotion, achieving smarter and more natural interaction between human beings and computers. The study of facial expression recognition has found important applied values.This paper first analyzed and summarized the common methods of expression recognition , then presented an facial expression extraction method by combining the two-dimensional discrete warelet transform(2D-DWT) with the independent component analysia(ICA) , the facial expression recognition algorithm based on optical flow and hidden semi-markov models in the presence of occlusion. The work are described as below:(1)A method of face features location based on the skin-color and geometry features is adopted.The face features location is a important step in the face expression recognition.The veracity of the face features location is a crucial target of a facial expression system. We used integral projecting to find the eyes and mouse by the scale of the face.(2)An efficient facial expression extraction method by combining the two-dimensional discrete warelet transform(2D-DWT) method with the independent component analysia(ICA) method is proposed. First,each image is decomposed into four sub-images by using the 2D-DWT approach,and then ICA approach is used to extract features form each sub-image respectively. Then, the differences of extracted features are obtained by subtracting features of neutral expression from the features of other expressions. All the differences of features are further combined and used for facial expression classification. Moreover,considering that the discriminative features extracted from each sub-image may not share the same metric scale measure,we also proposed an effective features combination method in this paper.(3)An facial expression recognition method based on optical flow and hidden semi-Markov models(HSMMs) in the presence of occlusion is proposed. A hidden semi-Markov model is an extension of HMM except each state can emit a sequence of observations, designed to remove the constant or geometric distributions of the state durations assumed in HMM. This method used the optical flow to extract the eyes and mouth features,and in recognition process,used the HSMMs to recognise the eyes occlusion and mousth occlusion and no occlusion,Experiments showed that HSMMs got better recognition rate in the presence of occlusion compare with the HMMs. (4)A prototype system of facial expression recognition is designed and implemented by object oriented technology. This system consist four modules that is face image preprocessing , face features location , facial expression feature extraction, expression classification. It proved the validity of our method.