Research And Implementation Of 3D Face Reconstruction Algorithm Based On Mobile Phone Video

Face is an important part of the human body,which plays a very important role in identification and emotional transmission.High quality 3D face model has many applications in life,such as virtual reality,face recognition,film and television production,games,auxiliary medical treatment,and so on.In recent years,image based 3D reconstruction technology is becoming more and more mature.With this technology,users can use the digital camera to capture face images and restore the textured 3D face models.This topic mainly uses the image based reconstruction technology to study how to reconstruct the 3D face for the video captured by the smart phone.We build a low cost,easy operation and easy popularization 3D face reconstruction system,use the intelligent mobile phone video shooting face ordinary light as input,an algorithm to extract key frames from the video,and then use the method of multi view stereo geometry to restore the real 3D face model,the main work is as follows:1.A key frame extraction algorithm for high parallax scene video is proposed.The key frame set selected by this method achieves higher accuracy and faster speed in 3D reconstruction.In this paper,we use the two constraints of image's baseline length and overlap rate to guide the selection of key frames,and propose the estimation method of image overlap rate under high parallax scene.Finally,we get a set of key frames with higher reconstruction accuracy.2.Optimizing the dense stereo matching algorithm to improve the accuracy and performance of the stereo matching algorithm.In order to avoid the influence of light,weak texture,shelter and other factors,this paper uses a variety of optimization methods to improve the accuracy and efficiency of the NCC matching method,and then optimize the minimum projection error of depth map among more than one adjacent images,finally get a relatively accurate depth map.3.Achieving an easy to operate and low cost 3D face reconstruction system.We use the video captured by smart mobile phone as input data,then select key frames from the video and use multi view stereo geometry method to reconstruct 3D face model.We achieve the key modules of 3D reconstruction system:the camera's position estimation,stereo rectification,dense matching,point cloud reconstruction and so on.