Smile Expression Recognition Based On Deep Convolutional Neural Network

Facial expression recognition is an important research topic in emotion analysis. As one of the basic expressions, smile plays an important role in expression recognition. Most facial expression recognition methods deal with frontal face images in constrained environment via one or more features, which not only inevitably results in some information loss to some extent, but also leads the methods be sensitive to face pose. scales transform and noise. Therefore, to meet the requirements of the practical application system, there are many key issues need to be further studied.Aiming at the face images acquired in unconstrained scenarios, this dissertation concentrates on smile expression recognition. The main contents and innovations are as follows:(1) After the newly-developed convolutional neural network and the traditional artificial neural network are comparatively studied, we make a further analysis on the realization principle of deep convolutional neural networks, which is different from the general structure of the multi-layer artificial neural network. There are three characteristics in the structure of deep convolutional neural networks, i.e. local connection, weight sharing and sub-sampling. Compared with other feature extraction methods, the convolutional neural network has automatic feature learning ability, which can effectively avoid the complexity of artificial design features. Additionally, digital images can be directly used as network input to avoid data reconstruction in feature extraction and object classification.(2) An image preprocessing method is proposed for face images acquired in unconstrained scenarios. First, in order to reduce complexity of subsequent image processing, original color images are converted to grayscale images. Then, mouth region localization is fulfilled according to 5 key points on the face, since mouth region is always playing an important role in expression analysis; secondly, mouth image brightness adjustment using the adaptive brightness adjustment, and the bicubic interpolation method on image size normalization. Finally, the experimental results show that the proposed method is more specific for the smile recognition.(3) An smile recognition method based on the deep convolutional neural network is suggested for face images acquired in unconstrained scenarios. First, the classical convolutional neural network model lenet-5 was improved; second, the improved lenet-5 model is constructed from a large number of training samples to automatically learn the multi-scale features of smile; Third, SVM is introduced to get a better classification performance instead of the original Softmax classifier. Finally, Experimental results on MTFL database and GENKI-4K database show that this smile recognition method is better than some other methods.