Research On Facial Expression Recognition And Automatic Annotation

In recent years,with the development of artificial intelligence and pattern recognition,facial expression recognition and related research works have been widely studied by scholars.It has great application value in business,medical treatment,public security,and other aspects.This dissertation focuses on the study of the recognition and annotation of the static facial expression image.The main research works are as follows:(1)In view of the shortcomings of Local Binary Pattern(LBP),Center-Symmetric Local Binary Pattern(CS-LBP)and Histogram of Oriented Gradient(HOG)algorithm in image feature description,Center-Symmetric Local Smooth Binary Pattern(CS-LSBP)and Histogram of Oriented Absolute Gradient(HOAG)are proposed.Combined with feature fusion algorithm,a facial expression recognition method based on the local texture feature and local shape feature is proposed.Firstly,face detection algorithm is used to detect the face region from the image.Besides,geometric change,scale normalization and filtering are applied to reduce the influence of disturbance factors such as posture,illumination and noise on recognition.Secondly,CS-LSBP and HOAG are used to extract two local features of facial expression image.Then,in order to give full play to the complementarity and discriminative ability of the two features,Canonical Correlation Analysis(CCA)is used to fuse two local features.Finally,Support Vector Machine(SVM)is performed for the expression classification.The Experimental results show that,the improved feature extraction method can extract the detail information of the image more completely and accurately.And the fusion method based on CCA can give full play to the representation ability of each feature.The facial expression recognition method proposed in this dissertation obtains a better recognition effect.(2)In view of the deficiencies in the construction of facial expression database,a method of automatic facial expression image annotation based on convolutional neural network is proposed,and a facial expression database in natural scenes is constructed to satisfy the data requirements of facial expression recognition.This method firstly uses the tool of image batch download to download a lot of facial expression images from the network,and the face detection and image preprocessing is executed on the collected images.Secondly,in order to solve the problem of unbalanced samples,data augmentation is carried out on the training set,and the augmented training set is used to train the convolutional neural network.Then,the original image and its mirror image of samples are sent into the trained neural network,and the output results of both of them are fused at decision level to improve the final annotation effect.Finally,according to the set of screening rules,the sample is labeled and the preliminary quality of the annotation is evaluated.The experimental results on RAF-DB(Real-world Affective Faces Database)indicate that,the data augmentation method and decision level fusion method in this dissertation can greatly improve the effect of facial image annotation.The manual evaluation and experimental comparison of the constructed database show the usefulness of the constructed database and the effectiveness of the proposed annotation method.