A Speech-driven Lip Animation Synchronization Method

In recent years,generating natural-looking 3D lip animation synchronous with input speech signal has been a challenging problem in computer graphics.At this stage,there are five kinds of methods to generate lip animation,i.e.procedural methods,physically based methods,data driven methods,machine learning based methods and other methods.Some of the above methods need to gather large quantities of data,some may result in high time consumption at runtime,while others may generate low accuracy results.An algorithm which doesn't need large amount of data,is efficient at run time and can generate high accuracy results is presented.This algorithm can be split into two phases,i.e.preprocessing phase and runtime phase.During preprocessing phase,46 English phonemes is classified into 15 classes.8 canonical lip poses are selected afterwards through Principle Component Analysis to form a canonical lip pose set.An animator is then required to design lip animation segments corresponding to phoneme sequences with different length.During runtime phase,an optimal sequence of lip animation segments is found to match the phoneme sequence of the input speech signal.These lip animation segments are then stitched together to generate the final result.Our method can be applied to several fields such as 3D movie production,game production and virtual reality,etc.A series of experiments are performed to analyze the time consumption and accuracy of our method.Experimental results show that our method has relatively high efficiency,and can generate lip animation with high accuracy in a short period of time.