Research On 3D Human Body Point Cloud Real-Time Reconstruction System Based On Multi-View Depth Perception

The three-dimensional reconstruction is a technique that acquires scene information through various types of sensors,processes and analyzes the acquired scene information by using computer related technologies,and then builds a three-dimensional model of the scene.In recent years,3D reconstruction technology has been widely used in many fields such as virtual reality,industrial safety production,object recognition and life service.However,the three-dimensional reconstruction method based on monocular cameras has some shortcomings and problems:firstly,it needs to move the camera to obtain scene information of different perspectives,which increases the time complexity of the system to a certain extent.Secondly,it usually only applies to 3d reconstruction of static scene.In view of the above problems,this paper studies a three-dimensional human body point cloud real-time reconstruction system based on multi-view depth perception.The main work of this paper is as follows:1)Analysis and selection of the calibration method of the Kinect v2 camera.Firstly,this part first elaborates the basic knowledge of camera imaging model and camera calibration,and then briefly introduces the traditional color and depth camera calibration methods,and analyzes the problems,advantages and disadvantages of various calibration methods in detail.Then,aiming at the problem that Zhang Zhengyou's calibration method can't detect the common checkerboard pattern captured by Kinect v2 depth camera,two different calibration checkerboard patterns are designed to verify the validity of the Kinect v2 depth camera calibration method used in this paper..Finally,the calibration method and calibration process of the multi-Kinect v2 depth camera are analyzed in detail.2)A depth image restoration algorithm based on sequential filling framework is proposed.Firstly,this part analyzes the reason why Kinect v2 generates invalid points,and then briefly analyzes the shortcomings of traditional deep image restoration methods.Finally,in view of the shortcomings of traditional methods,this paper proposes a depth image restoration algorithm based on sequential filling framework.The method firstly performs the coordinate transformation of Kinect v2 color and depth image,and then performs the invalid point marking in the depth image.Then,the invalid priority evaluation is implemented based on the method proposed in this paper.Finally,the hypothesis verification method is combined with the color image to complete the prediction of the invalid point depth value.3)The generation and registration method of multi-view point cloud is designed and implemented.This part first introduces in detail the generation process of point cloud and the method of 3D human point cloud segmentation—threshold method and Kinect v2 human bone flow method.Secondly,it describes the basic transformation relationship of 3D point cloud registration—rotation and translation,Then,the traditional point cloud registration method-based on normal distribution method and sampling consistency method is introduced briefly.Finally,aiming at the problem of insufficient real-time of traditional point cloud registration methods,this paper designs and implements ICP algorithm based on calibration parameters,and compares the above methods through experiments.4)The design scheme and experimental results of the system are given.This part firstly analyzes the selection of depth camera and the structure and configuration of hardware.Secondly,the specific implementation of multimodal image acquisition,image preprocessing,data communication,camera calibration,3D point cloud generation and registration,and 3D visualization are elaborated,and then on this basis,the overall technical roadmap of the system is constructed.Finally,the real-time performance of the system and the reliability of the system are analyzed.The experimental results show that the three-dimensional human body point cloud real-time reconstruction system based on multi-view depth perception can better achieve the 3D point cloud model of the target human body in real-time reconstruction.Compared with the traditional 3D reconstruction method,the proposed method not only achieves the purpose of real-time reconstruction,but also improves the partial missing of the reconstruction model due to scene occlusion.