Research On Feature Point Detection Method Of Facial Dynamic Augmented Reality

Facial dynamic augmented reality refers to the application research of real-time facial expression registration and posture estimation as the main object in order to realize the superposition of virtual and real face information under natural conditions.Many of these types of application research,such as the mobile virtual make-up sales platforms which aim to enhance the user's face,as well as face retouching and face animation all have a high demand for facial expression registration and posture estimation.In the augmented reality application research of facial dynamics as the main feature,facial feature point detection method is the foundation and key of virtual and real face matching and fusion.Over the past decade,face feature point detection method under the constraint condition has been extensively and deeply studied,and a lot of achievements have been made.In these results,it is not difficult to find that the detection rate and localization accuracy of the existing methods under natural conditions can not meet the needs of face localization and virtual face integration in the practical application for the face area.In this context,this dissertation has carried out research on face feature point detection in the case of face localization with large facial posture,complex facial expressions and so on,which provides a solid foundation for the application of the face dynamic augmented reality system under unconstrained conditions,and further improves the rendering effect of the virtual reality face fusion under the natural condition.The main challenge of facial feature point detection in the context of the above application comes from the following: There exists a big difference of face appearance and local texture between people who comes from different races,gender and ages.In the natural environment,some of the face images colleceted are occluded,therefore the face shape model trained after extracting features from the occluded images are not stable.And extracting features around facial feature points causes the problem of high-dimensional feature accumulation.In view of the above four difficulties,this dissertation focuses on the modeling of face shape based on sparse coding,facial point detection for faces under large face poses and complex expressions,as well as the compression process of high-dimensional feature model.The main contents and innovations of this dissertation are as follows:1.A method of facial feature point detection based on sparse constraint is proposed.Based on the idea of sparse coding,the proposed method uses the learning framework of cascade regression,and uses the sparse constrained reconstruction model to iterative search for the position of the facial feature points.The proposed method first learns a stable sparse feature dictionary through a large number of image data,and then the dictionary is combined with facial texture information for feature coding to further abstract features.Finally,a regression funtion is learned from the coded facial features to the increment of the face shape coordinate.2.A method of facial feature point detection based on recursive shape reconstruction is proposed.Considering the difficult problem of facial feature point detection that many face images in the natural conditions exist with facial expression and posture changes and partial occlusion,we propose a robust method of face shape sparse reconstruction.This method not only has sparse constraints on the local features of human faces,but also has global constraints on the whole face shape.This method effectively combines the learning of local facial features with the study of global face shapes.In order to make the method achieve better detection effect,this dissertation also adopts the strategy of combining multi-initialization and multi-parameter in the learning of model parameters.This makes the proposed method achieves a good detection accuracy in the natural conditions of large postures and complex facial expressions.3.A facial feature point enhancement method for model compression is proposed.In the above two face feature point detection methods,they are designed to extract a large number of features from the face image and learn models from these features in order to achieve better detection results.At the same time,in the case of dealing with posture changes and partial occlusion,it is necessary to maintain a moderate training model size and computational efficiency.In view of the above requirements,this dissertation proposes a facial feature point enhancement method for model compression.In this method,the number and location of predefined feature points in the database are selectively compressed,and model learning is performed on the compressed facial shape.Through the regression approximation of the sparse coefficients corresponding to different number of facial feature points,the corresponding regression mapping function is solved and the complete facial shape is reconstructed to realize the purpose of enhancing the facial feature points.4.A digital “Sichuan Opera Face Changing” based on facial dynamic is completed.Combining the facial feature point detection method based on sparse coding and the facial feature point enhancement method for model compression,a detection method based on face dynamics is proposed.This method aims at solving the problem of virtual reality face fusion in the process of digitizing “Sichuan Opera Face Chaning” in the complex facial environment such as large facial postures,complex facial expressions and potential partial occlusion.In this dissertation,the main study object is the human face area under natural conditions.And the robustness,detection rate and localization accuracy in the pratical augmented reality application are improved to a certain extent by applying the proposed methods.This study has a cetain role in promoting the pratical application research of face dynamic related augmented reality.