Research On Facial Expression Modeling And Mapping Technology Synthesis

With the rapid development of virtual reality and image processing technology,facial expression transplantation technology has attracted the attention of image and video little by little.This thesis mainly focuses on the use of CNN,RNN and expression mapping in expression transplantation technology based on feature point difference vector.The extraction of feature points is the key part of expression transplantation,compared with the traditional cascaded regression algorithm,the convolutional neural network could avoid the positioning error caused by the initial face feature when extracting facial features,but the current convolutional neural network is generally constructed with the trend of deeper network to extract more accurate feature points,which will lead to greater computational complexity.In addition,the convolutional neural network can't predict the characteristics of temporal correlation,such as head and eye feature information,while the recurrent neural network can extract these.Therefore,this thesis focuses on how to fully combine the advantages of both to solve the problem of extracting temporal correlation and non-temporal relevance features simultaneously.To achieve the effect of expression transplantation,the extracted 3D feature data are reconstructed into face triangular mesh,and the deformation coefficients for controlling the facial animation are calculated according to the triangular mesh.Aiming at the problems in the whole implementation process,this thesis introduces a lightweight structure into the standard CNN.Besides,a new hybrid structure is proposed to solve the problem of how to combine the two models.At the same time,an expression transplantation algorithm is proposed to achieve the effect of transplanting the initial facial expression to the face of an avatar.Due to the lack of public 3D face datasets,this thesis has labeled the datasets for training and testing models.The experimental parts demonstrate that our algorithm is able to quickly achieve a more realistic tracking effect.The main contents of this thesis are as follows:1.This thesis summarizes the principle methods of face tracking technology and the research results in recent years.The basic flow of face tracking technology is expounded,and the existing facial feature extraction technology,feature information extraction technology of head and eyes,and research status and improvement of data mapping technology are analyzed and studied.2.Through the analysis of face feature extraction technology,this thesis designs a lightweight convolution neural network to extract 3D feature points.The model is based on separable convolution structure,which can provide more accurate features with fewer parameters and greatly lower the computational complexity.In order to enhance the accuracy of feature extraction,a face detection method based on inverted triangle structure is used to detect the face bounding box of the image in the training set before extracting features from the model.3.Considering the authenticity of expression transplantation,this thesis builds a composite model based on LW-CNN model,namely Hybrid Recurrent Convolutional Networks(HRCNs),which takes into account the accuracy and rapidity of LW-CNN in extracting non-temporal correlation information and the effectiveness of LSTM model in processing temporal correlation information.To verify the effectiveness of the algorithm,this thesis evaluates the accuracy of the algorithm from two perspectives,namely,NME and RMSE.The final results demonstrate that our method can extract more accurate and stable feature points.Considering the wide application of the algorithm,this thesis uses the expression mapping algorithm based on feature point difference vectors to realize expression transplantation.Finally,the mapped face can show more subtle expression changes clearly.4.Due to the lack of public 3D face datasets,this thesis labels the 3D face datasets for training model.The label file includes 68 feature points of face,left and right eyes and head posture feature information.Finally,about 40,000 face data are prepared.