Research On Facial Expression Recognition Based On Fused Feature Extraction And Discrete HMM

For more than 20 years, facial expression related processing by computer has been an attractive issue in the areas such as computer vision, computer graphics, pattern recognition, etc. It can be widely applied into video conference, movie production, intelligent human computer interface, etc. On the basis of analyzing the existed methods carefully, we propose the texture information feature extraction method based on discrete wavelet transformation-orthonormal non-negative matrix factorization (DWT-ONMF), and the geometrical shaped information extraction method based on active appearance model(AAM), then fuse these two features by canonical correlation analysis(CCA), which is used for the observation vectors of discrete HMM to recognize the facial expressions. The main work is listed here:(1) To overcome the problem of NMF on the factorization time and the redundancy of basis matrix used in the facial expression feature extraction currently, the combination of DWT and ONMF is proposed in this paper. DWT is firstly used to extract the low frequency information that includes most of energy of a face image, then do the ONMF for it, so we can obtain the orthonormal feature space to extract the texture features through the projection of image sequences.(2) A new method based on texture information feature and geometrical shaped information feature is proposed to extract the hybrid feature. DWT-ONMF is used to extract texture features; AAM method is used to locate the position of the 68 points in the image sequences, 16 key points in the eyebrow, eyes, nose and mouth areas were selected, then the differences of coordinates between expression frames and the neutral frame were computed to extract the geometrical shaped features.(3) A facial expression recognition method based on hybrid features fused by CCA and DHMM is proposed. To overcome the potential information redundancy between texture features and geometrical shaped features, CCA is used to fuse these two features, then the fused feature is used for observation vectors of discrete HMM to train the 6 typical facial expressions. The proposed method uses correlation features of two groups of feature vectors as effective discriminant information, so it not only is suitable for information fusion, but also eliminates the redundant information within the features, which can achieve a good result use the DHMM as classifiers.(4) A prototype system of facial expression recognition based on fused feature and DHMM is designed and implemented. The modules in this system mainly include DWT, ONMF, some operation about AAM, CCA and DHMM. It can be proved that our algorithm is effective.