3D Reconstruction And Motion Capture Based On Depth Sensor

3D reconstruction has always been one of the hot topics in the field of computer vision and graphics.Its purpose is to effectively build the 3D model of the object.Nowadays,virtual reality(VR)and augmented reality(AR)are booming,and 3D reconstruction is the basis of them.It is an unavoidable problem for researchers to create 3D models quickly and effectively.With the development of hardware capture system and algorithm,3D reconstruction technology has developed from offline,non real-time and static reconstruction to online,real-time,dynamic and semantic reconstruction.What's more,the application of UAV opens up more possibilities of 3D reconstruction.It extends the scope of 3D reconstruction to dull,dangerous and inaccessible areas.It expands the application of 3D reconstruction and greatly improves its application value.This paper focuses on reconstruction of static scene and dynamic human motion capture.The main works are as follows:Firstly,we use a monocular depth camera for static scene reconstruction.Based on the method of depth map fusion,the 3D model is represented by voxels.After estimating the camera pose of the current frame,the point cloud of the current frame is fused into a single global volume model,and finally the complete surface is obtained.Considering that the large-scale reconstruction ability can benefit UAV 3D reconstruction,in this paper,the voxel hashing method is also used.This method allocates space only on the observed surface of the object,organizes the space structure by a hash table,realizes the high compression of space,and makes the large-scale scene reconstruction possible.Secondly,this paper discusses the new combination of virtual reality technology and 3D reconstruction,and develops a head-mounted-display-based telexistence system.Based on 3D reconstruction,the system aims to create a natural and friendly interaction interface between human and UAV,so that users can control all six degrees of freedom of the UAV freely and naturally via a VR helmet,and get a high-precision and more complete 3D scene model.At the same time,this paper builds two simulated telexistence and 3D reconstruction systems,and conducts two user studies,which verify the effectiveness of the proposed interaction interface in subjective evaluation and objective indicators.Finally,for the dynamic motion capture,this paper combines the volumetric dynamic reconstruction with the data-driven template registration method to reconstruct the geometric details of the human body using a single depth camera,and capture the non-rigid transform and the embedded skeleton shape.The outer layer is the observable human surface,and the inner layer is the embedded skeleton of human body.The two-layer representation solves the occlusion problem in the motion capture and surface reconstruction.The inner skeleton and outer surface are constrained to optimize each other,which makes the motion tracking more robust in challenging scenes.