Research And Application Of Facial Expression Recognition Based On Integrated Convolutional Neural Network

Facial expression recognition has always been an interesting and challenging problem.In practical applications,it is easily affected by various factors,such as lighting,posture,facial occlusion,age,ethnicity and other factors.According to research,traditional hand-extracted features cannot solve various factors that are not related to facial expressions.To solve this problem,this paper uses Convolutional Neural Network(CNN),which can input raw data and combine feature extraction and classification.The model has tens of millions of parameters that can handle a large number of training samples,and is a powerful automatic feature extractor.Current state-of-the-art algorithms show that using CNN ensemble can outperform a single CNN classifier.Therefore,this paper conducts research on optimizing the structure of convolutional neural networks and learning the optimal ensemble rules of basic classifiers to improve the accuracy of facial expression recognition.The main work and innovations are as follows:1.Inspired by VGG Net's neat structure and Xception structure,three different structured sub-convolutional neural networks were designed to ensure the complementarity of the network.The compact and effective network design makes it sufficient to complete the task and easy to train.By introducing the global average pooling layer instead of the fully connected layer,the amount of model parameters is reduced,and the model recognition rate is guaranteed.The introduction of a deep separable convolution module with 4 residuals can further reduce the network parameters while deepening the network.In addition,batch normalization and L2 regularization are added to the network to solve the over-fitting problem and improve the network generalization ability.Perform experiments on the CK+ database and the FER-2013 database,and each sub CNN can complete the expression recognition task better.2.It is proved through experiments that the use of ensemble methods usually performs better than any single classifier,so an expression recognition method based on ensemble convolutional neural network is proposed,and the most effective ensemble rule is found.In combined basic learners,weighted average is better than majority voting and simple averaging methods.Stacking generally has better performance than Bagging,SAMME,and Snapshot.This is because meta model training in Stacking is more effective than basic learner training.Efficiency,so Stacking achieves a good balance between performance and efficiency.3.Inspired by ensemble ideas,a real-time facial expression recognition system was established by adding decisions on the recognition results of eyes and mouth areas.First,the eye databases and mouth databases were used to train the best models through the above three sub-networks.In the real-time facial expression recognition process,three models were called and the majority voting method was used to improve the system performance.In addition,a visual interface of the recognition system was constructed using PyQt5.