Key Techniques Study On Facial Expression Recognition

Communication in any form either verbal or non-verbal is vital to complete various daily routine tasks and plays a significant role in life. Facial expression is the most effective form of non-verbal communication and it provides a clue about emotional state, mindset and intention.Till now, facial expression recognition has been successfully applied to various fields such as safe driving, merchandise sales, clinical medicine, and so on. This thesis explores key techniques related to facial expression recognition. The main work and contributions are as follows.(1)Static Facial Expression Recognition based on WPCANet and Block Weighted Histogram.By combining unsupervised learning way for convolution filter banks with multi-stage feature detection strategy of Convolutional Neural Networks(CNNs), a WPCANet model is built and used for unsupervised feature extraction in static facial expression recognition. With the incorporated category information about training samples, a weight map based on FDR(Fisher’s Discriminant Ratio) is generated, and then a block-weighted histogram from the output stage of WPCANet is produced for the representation of facial expressions. Finally linear C-SVM classifiers are adopted and integrated to implement facial expression recognition of seven kinds.Experiments on JAFFE dataset demonstrate the effectiveness of the proposed static facial expression recognition algorithm.(2)Dynamic Facial Expression Recognition based on Multi-Visual and Audio Descriptors.A dynamic facial expression recognition algorithm based on multi-visual descriptors and audio features is proposed under unrestricted conditions, in which dynamic facial feature extraction was conducted based on local spatial-temporal feature representation via multi-visual descriptors. Furthermore, the combination of video and audio features improves the recognition performance. Dynamic time warping based on timeline segmentation and covariance matrix proves to be effective in analyzing dynamic expression sequences of different time duration.To improve the generalization performance of facial expression recognition model, an integrated decision-making strategy based on weight voting by multiple individual recognition models is introduced. In order to effectively learning the weight for each individual recognition model, the method of voting weight learning by random re-sampling and the method of voting learning based on comparative advantages of individual recognition model are proposed.Finally the above ensemble model is applied and the recognition performance is further improved.Experiments on AFEW5.0 dataset validate the performance of the proposed dynamic facial expression algorithm.