| Facial expression analysis is a hot research topic in the field of computer vision,whose purpose is to judge the expression state of the target according to the input image.Due to age,skin color,gender and other factors,the facial images of different individuals will show different characteristics,and factors such as face posture,lighting conditions and visual occlusion make it difficult to judge facial expressions.Existing facial expression recognition methods based on deep learning usually use specific algorithms to extract the facial part of the input and use convolutional layers to extract high-dimensional features of the image,and then make classification judgment through Softmax regression.These methods usually use a large number of convolutional layers to obtain more comprehensive features when extracting facial features,making the algorithm complexity too high,and high network depth will lead to network degradation.In addition,the existing methods have insufficient preprocessing work,which requires the network to observe the whole image,and cannot pay more attention to the key detail areas that affect the expression.In this dissertation of thesis,we propose two optimized network structures and construct a real-time facial expression analysis system.The specific contents are as follows:1.Aiming at the solving the problem of heavy computation and network degradation caused by a large number of convolution layers in neural network,a facial expression analysis network based on spatial attention mechanism is proposed.The network introduces the structure of Res Net-18 as the basic feature extraction module.After getting the feature vectors of each layer,they are respectively sent to the spatial attention module to distinguish the importance of each facial region for expression recognition.In addition,in order to solve the problem of uneven distribution of the number of samples in each category of the dataset,the focal loss function is introduced to make the network learn more difficult samples.The experimental results show that the overall recognition accuracy of facial expression analysis network based on spatial attention mechanism has been improved on CK+ and FER2013 expression classification data sets,which is at the leading level at present.2.Aiming at the solving the problem of insufficient facial image preprocessing,a facial expression analysis network based on facial landmarks is proposed.The network detects 68 facial landmarks during image preprocessing,and segments facial images according to the key points.In feature extraction,the convolutional network based on inverted residual blocks is used to extract image features and the graph convolutional network is used to extract structural features,which makes the feature representation ability stronger.Finally,the channel attention mechanism is used for feature fusion.The experimental results show that the overall recognition accuracy of facial expression analysis network based on facial key points is further improved on CK+ and JAFFE expression classification datasets.3.The facial expression real-time analysis system is based on Python.We use Py Qt toolkit to design the graphical interface and use the facial expression analysis network based on facial landmarks as the core algorithm.The system supports the input of a single image or a real-time input image from the camera,and can analyze the expression state in real time.By designing different scenarios,the system is tested under two input modes.The results show that the system has generalization and availability,and meets the real-time requirements in conventional scenarios. |
