Study On Data Processing And Animation Reconstruction Based On Facial Motion Capture

Facial motion capture is an emerging branch of the motion capture (MoCap) field, which is a science of capturing facial motion by MoCap data acquisition method, processing and analyzing MoCap data, then generating facial animation from the processed data. Facial MoCap is a multi-disciplinary intersection and penetration of human body engineering, computer graphics, image processing, data processing and so on, and is the current research focus in computer science. Facial MoCap is an important issue with not only theoretical significance but also application value. The facial MoCap has been widely used in modern film and television animation, game production, medical analysis, virtual reality and other fields. This thesis systematically studied several key issues in facial MoCap, and made some academic achievements, at last, a prototype system for facial MoCap data processing and facial expression reconstruction was developed. The main work of this thesis includes the following aspects:1. Automatic facial template matching based on spatial geometric flexibility:According to two facial templates of identical cardinality with global similarity but local non-rigid deformations and distribution errors, the proposed method used heuristic methods to normalize the templates, the motions of local markers to correct local match, and the temporary feedback method (TFM) to improve reliability of the match, then achieved an automatic process to match the templates from local to global. The experiments proved that the method is robust, and it can fast and effectively match the different templates under different facial expressions in a deterministic and automatic process, which resolved the requirements of troublesome manual interventions during template generation.2. Online data processing and trajectory smoothing for facial MoCap:According to the data missing and noise of raw MoCap data, an online processing method based on dynamic spatial-temporal information was proposed. First, an analysis of noise propagation problem is proposed, and a noise propagation solution module (NPS) based on adaptive Kalman filter is used to suppress the noise propagation. Second, a 3D facial topology-based sophisticated non-rigid motion interpreter (SNRMI) is put forward, together with a dynamic tracking method, which could not only track the valid non-missing data effectively but recover several adjacent markers under long time missing. Third, to rule out wrong tracks generated from the markers in open structures (such as mouth, eyes), a semantic correction method is proposed. Lastly, an online curve modeling method is proposed to construct curves online to smooth marker trajectories. Experiments show that the method can automatically process raw data with noise and long time missing problems, and can simultaneously smooth trajectories.3. Simulation of facial animation based on facial MoCap data:Based on the idea of driving facial model from different functional regions, a cross-mapping algorithm based on radial basis functions (RBF) method is constructed for the generation of motion data for different facial models. During model driving, virtual markers are added and their motion data are computed, then, RBF is used to drive the different personalized facial models to generate realistic facial animation. A pre-computing algorithm is proposed to reduce computational cost during real-time simulation. The experiments proved that the method can not only map the MoCap data of one subject to different personalized face but generate realistic facial animations.4. Development for the prototype system of facial MoCap data processing and facial animation reconstruction:The system is designed and positioned to a interactive graphics system with software intellectual property rights, the proposed algorithms of the thesis are integrated, a unified underlying data structures is built, a friendly and powerful interactive interface is provided. To make the MoCap data processing and facial expression simulation intuitive and efficient, the system seamlessly integrates the three main modules:template construction, data processing and smoothing, and model-driven process. Practice has shown that the system can effectively prove the proposed methods, and process MoCap data and reconstruct facial expression animation by full interaction, moreover, the processing is efficient and the simulated animation is realistic.To sum up, several major issues in facial MoCap data processing and facial animation. reconstruction have been studied, and under the support of the proposed algorithms, prototype system for facial MoCap data processing and facial expression animation reconstruction has been designed and developed. With a great deal of experiments, the effectiveness, robustness, credibility level and efficiency of each algorithm were verified and proved.