Design And Construction Of Handheld Wireless Real-time 3D Reconstruction System Based On RGB-D Camera

The 3D reconstruction can be divided into dense reconstruction and sparse reconstruction according to the type of reconstruction model.Currently,the monocular reconstruction algorithm can realize real-time sparse reconstruction.However,when the scene is densely reconstructed,the time consumed is often large.And when the 3D reconstruction can be realized on handheld devices,the application scenario of 3D reconstruction will be broadened.This paper design and implement a system for realizing 3D reconstruction in real time by handheld mobile terminal.The whole system is divided into the real-time reconstruction of 3D point cloud model mobile on handheld device and real-time reconstruction of 3D implicit surface model on computer.The implement includes the following aspects:First of all,we build the hardware device of the mobile based on the Raspberry Pi and RGB-D camera,and we calibrate the camera parameters and correct the distortion.The depth and color image are acquired by the calibrated camera,and at the same time,the data is transmitted to the computer in 0.15 s to prepare for the 3D implicit surface model reconstruction on the computer.Second,the construction of 3D point cloud model is implemented on the mobile device.Using the collected depth and color data,feature extraction and feature matching are performed based on the ORB algorithm,and the bundle adjustment is applied by graph optimization to improve the matching precision.During the reconstruction process,key frames and closed-loop optimization are introduced to improve time efficiency.Third,the real-time 3D implicit surface model is constructed based on the transmitted data on the computer.The reconstruction process is performed by blocks from local to global.We implement feature extraction and matching by SIFTGPU.Then,select keypoint pairs between frames by covariance as well as select matching pairs in the block based on projection area of sparse features and residuals between dense point clouds.Implement local optimization using the above filtered data based on sparse optimization and dense optimization.Finally,reconstruct a 3D real-time implicit surface model based on the TSDF model.Above all,3D real-time sparse point cloud model is constructed on the handheld device,and 3D real-time dense implicit surface model is constructed on the computer to realize a handheld wireless real-time three-dimensional reconstruction system.