Mobile 3D Reconstruction With Inertial Measurement Unit Data

Three-dimensional reconstruction is a popular research field in computer vision,and has been widely used in computer-aided design,game,military,virtual reality,etc.The image-based 3D reconstruction technique has been widely studied at home and abroad due to its strong applicability and low cost.Among the current mainstream reconstruction algorithms,the Structure-From-Motion(SFM)is the most mature.However,since the traditional SFM algorithm has a large time complexity and the reconstruction result is ruined by few input images,this paper proposes an advanced SFM system with Inertial Measurement Unit(IMU)data.It aims to reduce the dependence on the input image,reduce the time complexity and improve the reconstruction results.And this paper also proposes two methods to filter the point cloud.The detailed contributions of this paper are as follow:(1)We propose an advanced SFM algorithm with fusion of IMU.The system first reads the IMU data of the mobile phone when taking pictures,and after preprocessing the data with Kalman filter,it calculates the camera rotation matrix of each image according to the data.Then we propose a method based on the Lie algebra theory to transform the coordinate system.Finally,this paper proposes to combine the rotation matrix calculated from IMU with the Bundle Adjustment(BA)in the traditional SFM algorithm.It is effective to improve the results and reduce run-time of the program.(2)We propose a denoising algorithm based on the camera coordinates obtained during the reconstruction process to remove background noise.Background noise is generally composed of ambient noise and ground noise.First,a cylinder wrapped around the object model can be obtained.The point outside the cylinder is considered as ambient noise.Our denoising algorithm projects the point onto the plane where the camera is located and determines the point is ambient noise if the projection is outside the cylinder.In addition,for the ground noise,our method calculates the scalar product value of the camera plane's normal vector and the point vector.The point is the ground noise when the scalar product value is maximal.(3)We propose a denoising algorithm based on the region growing of cameras' center to remove outlier noise.We solve the center of the cameras.And the closest 3D point to this center is considered as the initial seed point.Then a region growing algorithm is used until there is no new seed point in the neighborhood of the points.In this process,the points that have not been traversed are outlier noise.In summary,the validity of the proposed method is verified by theoretical analysis,and various experimental results have indicated our method's excellences over other existing methods in both qualitative and quantitative aspects.In addition,in the denoising experiment,we combine our method with the current mainstream SFM reconstruction system,which shows that our method has certain compatibility.