3D Face Reconstruction Via Fusing Deep Convolutional Network And SFM

3D face modeling is a hot research and application topic in the field of computer vision and human-computer interaction.With the development of science technology on artificial intelligence,the 3D face modeling is more diverse.The face feature information acquisition of most 3D face reconstruction algorithm methods always need shooting faces in fixed postures,it lead to limit their application range and convenience.The rigid structure from motion algorithm can recover object shape from the video sequences without 3D prior information,so 3D human face sparse structure can be also reconstructed by this method,but the human face is non-rigid object,the number and the accuracy of face feature points we extracted would have an important impact on the robustness and reality of final reconstruction model.In the field of 3D reconstruction from 2D face image sequences,the feature points extraction result is impeded by the problems of the occlusion and illumination or pose variations frequently.Different from 3D morphable model fitting algorithm,the rigid structure from motion reconstruction method based on face feature points puts forward higher requirements to the robustness and accuracy:firstly,the subject investigated of structure from motion algorithm are feature points sequences,so the reconstruction result based on SFM method is sensitive to the accuracy of feature points;secondly,the sparse structure matrix solved by rigid structure from motion algorithm is an approximate solution;thirdly,the sparse structure feature points also have an important impact on the process of deforming sparse structure into dense model.These three questions show that the accuracy of feature points could lead to the performance of final on a large extent.In recent years,this algorithm has been improved,on this basis,deep convolutional network algorithm has been fused for structure from motion algorithm and overcomes the shortcomings in 3D face modeling.Specific improved parts are shown as follows:(1)Fusing the task-constrained deep convolutional network algorithm for extracting face feature points.The face feature points can be extracted more accurately than traditional methods under the different scene including occlusion and illumination or pose variations.(2)The rotation matrix would be swapped to orthogonal matrix difficultly as the face sequences increase.In order to solve this question of the traditional factorization for structure from motion algorithm,one modified matrix is imported according to the property of rotation matrix(3)A re-registration method by matching the nasal tip of 3D face sparse points cloud and general 3D face model is presented to directly against the feature points sensibility question of the reconstruction method based on feature points.All proposed algorithms in this thesis are tested on data sets to validate the effectiveness of the aforementioned methods.Finally,these methods are applied in 3D face modeling system to verify validity and feasibility of the system.