3D Head Snapshot System Based On Multiple Depth Cameras

In this dissertation,we build a 3D head snapshot system based on five depth cameras which can finish data capture in one second and then reconstruct geometry and texture robustly.As the name implies,3D snapshot is similar to 2D photography in daily life.It is a process of obtaining 3D representation of objects with fast geometry and texture reconstruction.To meet the requirements of fast data capture and reducing the dependence on the head motion,the original depth noise cannot be well eliminated in the process of depth fusion which can be done well in the case of rotating camera or rotating head.To solve this problem,we train a noise reduction network that can use depth image as input directly,which can effectively reduce the noise of the reconstructed geometric surface and improve the smoothness of the geometric surface.In order to make the reconstructed head do not have crack artifact at the joint of different camera data and improve the tolerance of the system to the slight change of the relative position of five cameras,we use global optimization method based on the depth frame to further improve the pose accuracy of each frame on the basis of the calibration external parameters.Before KinectFusion reconstruction,the original depth data is preprocessed according to the accuracy of the original measurements.About texture mapping,to tackle the problem that original multiple image based texture mapping algorithm which is sensitive to light condition and may be exist crack artifact on result texture,we add face landmarks prior and face contour prior to improve the mapping result.The noise reduction,global pose optimization and geometry reconstruction are all implemented on GPU in parallel.Through the combination of these parts,the system can finish head data capture in one second and obtain the results of geometry and texture reconstruction in 15 seconds.During the process of capture,the head is not required to move in a certain way and the person just only sit naturally.The process is similar to self-service photo taking and compared with the reconstruction using a single camera,the capture speed is accelerated and the dependence on user behavior is reduced at the same time which greatly improving the user experience.