Research On Indoor 3D Reconstruction Model And Method Based On The Data Of RGB-D Camera

The development of Geographic Information Science(GIS)is usually highly correlated with the development of surveying and mapping and computer informa-tion technology.On the one hand,surveying and mapping technology provides data sources for the study of GIS,on the other hand,the application of GIS also poses new challenges to surveying and mapping and computer technology.With the rapid de-velopment of the global mobile Internet and the update of sensor technology,GIS has entered a new stage of development:from serving the government to the general pub-lic;from the macro to the microcosmic;from the outdoor to the indoor and outdoor in-tegration.From the requirement of indoor geographic spatial data,the rising popular GIS applications put forward the challenges of scanning speed,equipment cost,porta-bility and computation density.Traditional laser scanning and oblique photogram-metry technology can no longer meet the requirements of indoor GIS applications.RGB-D camera is a new consumption level visual sensor,which is made up of RGB lens and depth lens.When shooting,the camera can get texture and depth images.Generally speaking,there are four main problems in the field of 3D reconstruction using RGB-D cameras:(a)Low power RGB-D cameras have the disadvantages of low resolution and large noise,for example,the real resolution of the Xtion Pro Live depth camera is only 320*240 pixels,far below the resolution of the ordinary camera or industrial camera;(b)It is necessary to obtain indoor scene data for many times,and reconstruct a typical room with data acquisition from thousands of views.In or-der to obtain a three-dimensional model,all these data frames need to be accurately registered(i.e.attitude calculation),and the registration of a large number of rough and depth data(point cloud)is a research problem that has been widely concerned and has not been solved,(c)Compared with traditional photogrammetry,there is no GNSS auxiliary signal in indoor scene.The bundle adjustment algorithm based on visual feature points generated by low resolution image data converges easily to local optimum.(d)Because of the large amount of data processing,it is a difficult problem to deal with the original image data in real time and achieve a good 3D model,but it is of great significance to the application of the algorithm.In view of the above prob-lems,this paper makes a thorough study from various aspects.The main research contents and achievements are as follows:(1)The uncertainty distribution of depth observation is deeply discussed.It is found that the key problem of rotation and translation estimation is affected by this noise.A decoupling algorithm for rotation and translation of RGB-D camera is pro-posed.Firstly,the rotation matrix is estimated by texture image information,and the rotation matrix is separated from the deep noise.Next,we further study the distribu-tion characteristics of the depth observation noise,and develop a Gauss mixture un-certainty description model based on multi-scale windows.Taking advantage of the characteristics of nonlinear and linear optimization,the algorithm improves the con-vergence stability and accuracy of the optimization.The experimental results show that the RMSE accuracy of the decoupled estimation algorithm is 24.2%higher than that of the traditional algorithm.(2)Aiming at the problem that 3D reconstruction algorithm with traditional point features is easy to fall into the problem of local optimal solution.In this paper,a submap fusion algorithm based on point and plane features is proposed.Based on the advantages of point and plane features,the submap generating algorithm based on three aspects of feature points,image patches and planes is studied.The process flow of sequential merging of local submaps to global maps is designed.It is found that the traditional Hessian-Normal plane parameterization method has an over pa-rameterized problem.Based on the manifold space,a new plane parameterization method is developed to solve the problem of over parameterization and local min-imum.The fitting experiment shows that the iterative efficiency of this method is 73%higher than that of the traditional method.In the submap fusion algorithm,the author use the properties of the covariance matrix of the plane parameters,and the author creatively solves the problem of plane data association(the problem of planar descriptor descriptors).Both qualitative and quantitative experimental results show that the proposed algorithm is better than the traditional bundle adjustment algorithm.(3)In order to further improve the ability of the optimization algorithm to recon-struction the 3D model,on the basis of the previous plane features,this study uses the deep neural network detection technology to extend the available information to the object level.An ORB-SLAM2 based point tracking module is proposed,and a joint observation constraint framework including point feature,object rectangle informa-tion and plane information is established.It extends the traditional bundle adjustment model which minimizing the re-projection error,and achieves a challenging indoor scene.In the experiment,the precision of our 3D model and the reconstruction model of the high precision laser scanner is compared.The result shows that the consump-tion level camera can also provide an indoor 3D model with high precision(average error<4cm).(4)Indoor 3D reconstruction not only provides the surface model of the scene,but also provides the location information of the camera.In order to solve the fact that it is difficult to use the 3D model to achieve real and immersive effect in the indoor scene,a multi RGB-D camera collaboration data acquisition platform is con-structed and a prototype system is built.First,the author discusses the advantages of multi RGB-D camera cooperative data acquisition,hardware configuration method and configuration scheme of ARM control acquisition.On the basis of analyzing the complexity of each module,the data processing framework of ARM and ordinary PC is put forward on the basis of the analysis of the complexity of each module.For the problem of large difference between the points,a solution based on wide baseline matching is designed and implemented.Based on the implementation of each mod-ule,a Web-based system is finally developed.The system makes full use of 3D model information,camera pose information(positioning information),panoramic texture image information,and provides users with a high immersion and real sense of indoor roaming experience.