3D Expression Animation Driven By Real-time Face Data

Today's society is developing to an intelligent society at a high speed,and human-computer interaction technology has also penetrated into various fields of society.Among them,the use of facial expression data to drive animation has been widely used in the fields of film and television,medical treatment,and security.Due to the high cost of using optical instruments to obtain three-dimensional face data,this paper uses a common optical camera to collect two-dimensional images of the face in real time,extract key features of the face,expression information and other facial features from it,and express it linearly with the 3DMM model and Based on the average face model generated by the BFM database,the neural network is used to estimate the deformation parameters of the average face from the face image,that is,the model is deformed through the parameters to obtain the three-dimensional data of the face.Based on the three-dimensional data,combined with the designed patch data,a 3D effect image can be generated through the renderer.Due to the extremely short processing time of a single sheet,continuous 3D image generation can be achieved to form a 3D animation.The main work of this article includes:(1)Image preprocessing.In order to avoid the influence of factors such as light and fog in the process of optical face image collection,a fast image enhancement method is proposed.For face detection,Opencv and Dlib methods are used to achieve fast face positioning and feature point acquisition,respectively.(2)Recognize facial expressions.Combined with CNN network,a kind of multi-convolution and multi-pooling neural network model is designed,and high recognition rate is achieved on FER2013 and JAFFE databases.(3)Realize face reconstruction.In this paper,the face alignment work and the face reconstruction work are coupled.Based on the 3DMM model,the face model is deformed by changing the deformation parameters,and the face alignment is changed by changing the projection matrix.Based on Mobile Net,a cascade network is designed to realize parameter estimation.A loss function combining multiple losses is designed to improve the network convergence performance.(4)Real-time 3D face animation generation.Real-time face images are obtained based on ordinary cameras,and parameters are obtained from the face images according to the face reconstruction method to achieve 3DMM model deformation.The model is mapped by the renderer to obtain 3D facial expression animation.The 3D face animation generation under complex lighting is analyzed.