Research On Real-Time 3D Reconstruction Using Smartphones

3D reconstruction using smartphones has the advantages of low equipment cost,portability,and real-time feedback.With the continuous improvement of smart phone computing performance,it is feasible to perform real-time 3D reconstruction on smart phones.However,the real-time 3D reconstruction algorithms on smartphones generally suffer from inaccurate pose estimation,low algorithm efficiency,and the size of the reconstructed scene.In order to solve the problems mentioned above,we design a system for real-time 3D reconstruction on smartphones.The system consists of three threads,namely: modeling thread,back-end optimization thread and display thread.The modeling thread is responsible for getting color images and initial poses,key frame selection,computing depth maps and point cloud fusion,the back-end optimization thread is responsible for closed-loop detection and pose optimization;and the display thread is responsible for model surface extraction and drawing models.The system obtains the initial poses in real time through the AR solution provided by the smartphone manufacturer.With the initial pose,the visual bag-ofwords model is used for loop closure detection and camera poses are globally optimized by the reconstruction system.After the pose optimization,fused models are optimized at the same time.Then according to the computing performance of the smartphone,we estimate depth maps with the DIS dense optical flow,and optimize depth maps with the bilateral filter solver,so that depth maps are smoother on the plane,and edges are closer to the edges of the color image.Based on the TSDF,we also fuse the RGB value into the model so that we can get color model in real-time.And the Voxel Hashing is introduced to enable the system to reconstruct a high-quality indoor scene model.We adopt Open CL to fully use the GPU performance of the smartphone to accelerate the parallel calculation of the algorithm.The experimental results show that our system can reconstruct high-quality 3D color models of indoor scenes on the smartphone in real-time,and provide real-time preview feedback.The reconstructed 3D model can be used for many aspects such as model display,scene display,AR application and so on.