Research Of Performance-driven Based Facial Animation System

Performance-driven based face animation is a kind of technique to drive the virtual characters to make the same head gesture and expression based on the performer's facial action.It can be widely used in the fields of film and television production,human-computer interaction,game production,teleconferencing and so on.In recent years,it has been a research focus of computer vision and computer graphics.At present,the mature performance-driven based face animation systems have relatively harsh conditions of use,some of them need complex and expensive sensing equipment as support and some of them need high-performance computer to complete high operations.Therefor they are difficult to promote to consumers.The purpose of this paper is to build a real-time,accurate,consumer-level performance-driven based face animation system that captures facial expressions using ordinary monocular cameras and redirects the captured information to the animated characters.There are two main modules in our system:face expression capture and facial animation synthesis.The main research and contributions include:1.An improved fast 3D face modeling algorithm based on optimization method is proposed.In order to remove the redundant information in the face model tensor for synthesis,it is proposed to compress the tensor by high order singular value decomposition(HOSVD).Aiming at the problem that the manual calibration errors affect the physical meaning of the residuals,different residual modules are established in the energy equation according to the calibration characteristics of different facial parts.Experiments show that the proposed method improves the accuracy of the algorithm.2.A 3D facial expression capture algorithm based on random forest feature extraction is proposed,which directly obtains the 3D face shape from the 2D image.This method solves the problem that the capturing device is complicated or the computational complexity is high in the existing system.In this algorithm,combined local binary feature extracted with random forest is used in linear regression to improve the precision of the system.The feature of shape indexed Normalized Pixel Difference(NPD),which is expressed by barycentric coordinate,is used as the basis of decision tree division in the forest,and it improves the robustness to different face pose.3.An animation generation algorithm based on triangular mesh deformation transfer and an animation generation algorithm based on Blendshape are proposed to drive targets with different 3D facial structures,which improves the efficiency and authenticity of animation.The incremental weight method is proposed to enhancement the animation stability and the Lagrangian interpolation method is used to improve the frame rate of the animation.These two methods effectively improve the problem of jitter and frame rate which are common in the performance-drive based facial animation system.