Mobile-based Real-time 3D Reconstruction Of Objects

In recent years,technology about 3D reconstruction of real-world objects based on the input of cameras or sensors has drawn more attention.With the development and wide use of mobile devices,mobile phone has changed people's lifestyle and has become indispensable in modern life.In the future,more work will be done on mobile devices,so real-time 3D reconstruction based on mobile devices has important application prospects.However,most reconstruction algorithms need the depth of images as input,which depends on the support of specialized equipment and cannot be implemented on mobile devices.Fortunately,with the studies of SfM(Structure from Motion)algorithm,some 3D reconstruction algorithms that do not require depth information have been proposed,but most of them need processing big data.But the graphics processor on mobile devices generally shows low performance,which makes it difficult to achieve real-time processing of such big data on mobile devices.There are also some researchers proposed non-real-time solutions based on mobile devices or transmit the captured images to internet and computing on-the-fly,but such solutions are not user-friendly.In this paper,we studied the existing 3D reconstruction algorithms.During 3D reconstruction of mobile device,we applied Apple ARKit to improve the stability of pose estimation for mobile devices.Besides,we proposed a layered projection method with high parallelism and improve the efficiency of model projection in 3D reconstruction.We design a real-time 3D object reconstruction method on mobile devices.The input of our algorithm is the sequence frames obtained by the RGB camera on mobile device.Firstly,we use the dense image rectification and stereo matching algorithm to estimate the depth map.Then,a parallel model updating algorithm is used to fuse the camera image and the depth map into a TSDF volumetric surface model.Meanwhile,the parallel model projection algorithm is used to provide real-time feedback to users.The experimental results show that the algorithm can reconstruct 3D model with completeness of surfaces on the mobile device in real time.In this paper,the performance deficiency of portable camera and portable processor on mobile devices is handled so that feedback can be provided to users in real time.The output volume surface models can be used in many scenarios like virtual reality and augmented reality.