Research On Large-Scene 3D Reconstruction Based On Image Acquisition System With PTZ And Handheld Method

With the development of robot navigation,autonomous driving,and virtual reality,the 3D reconstruction technology for large scenes has become increasingly important.As these new technologies enter a new stage of development,3D reconstruction services are becoming increasingly popular,3D reconstruction scenes are becoming more complex and diverse,3D reconstruction image acquisition is becoming more integrated,the efficiency of 3D reconstruction needs to be improved,and the cost of 3D reconstruction equipment needs to be reduced.Aiming at the challenges that new technologies pose to the efficiency,cost,accuracy,and stability of 3D reconstruction systems,the use of low-cost 3D reconstruction equipment to establish an efficient image acquisition system for high-precision 3D reconstruction is an urgent problem to be solved.The main research contents and results are as follows:Aiming at the problems of high cost,complicated acquisition process and low efficiency of the current 3D reconstruction equipment,a self-developed PTZ(Pan,Tilt and Zoom)and a handheld device are raised to establish an efficient and PTZ image acquisition system for 3D reconstruction.Among them,the self-developed PTZ image acquisition device is composed of multiple Xtions,which can automatically and efficiently acquire images of large indoor scenes.In addition to depth cameras and color cameras,handheld image acquisition devices are also equipped with IMU(Inertial Measurement Unit)and wide-field fisheye cameras.Multi-sensor fusion enables accurate pose estimation and is more suitable for 3D reconstruction of large scenes.The 3D reconstruction system based on PTZ and handheld image acquisition equipment has developed a panorama roaming and display for 3D reconstruction system,which can realize immersive browsing of the 3D reconstruction model,feel the shape and distance of the 3D reconstruction models.After experiments,it is proved that the image acquisition system based on PTZ and handheld image acquisition equipment proposed in this paper establishes three-dimensional reconstruction,optimizes process of the image acquisition,improves the efficiency of image acquisition system,and reduces the cost of three-dimensional reconstruction image acquisition equipmentAiming at the shortcomings of low efficiency of large indoor scenes 3D reconstruction,low accuracy of 3D reconstruction models,and poor display effects,a 3D reconstruction system based on PTZ image acquisition equipment is proposed.Camera calibration of self-developed image acquisition equipment was first studied.Since the image acquisition equipment uses a solution consisting of three Xtions,the internal calibration of the color and depth cameras of a single Xtion,the calibration of the external parameters of a single Xtion color camera and the depth camera,and the parameter calibration between multiple Xtions are thoroughly studied.Then the image feature extraction algorithms are analyzed on datasets and in theory.Aiming at the single-site and multi-site working modes of the PTZ image acquisition system,combined with the analysis and experiments of the GMS(Grid-based Motion Statistics)algorithm,the optimal algorithm combination is selected and the stability experiments and result analysis are performed.Next,the pose estimation algorithms are analyzed,and combined with the single-site and multi-site unique pose optimization algorithms and loop detection algorithms of the PTZ image acquisition system,the accuracy of pose estimation of the PTZ 3D reconstruction system can be effectively improved.Next,the pose estimation algorithms PnP and ICP are analyzed,and combined with the single-site and multi-site unique pose optimization algorithms and loop detection algorithms of the PTZ image acquisition system,the accuracy of pose estimation of the PTZ 3D reconstruction system can be effectively improved.And in the 3D reconstruction of large scenes,the improved TSDF(Truncated Signed Distance Function)algorithm is analyzed and used,and experiments have proved that it can effectively improve the 3D reconstruction system based on PTZ image acquisition equipment efficiency.Finally,a large number of experiments can prove that 3D reconstruction based on self-developed PTZ image acquisition equipment and systems can effectively improve the accuracy and efficiency of 3D reconstruction.Aiming at the problems of visual odometry with camera shake,fewer feature points in the modeled scene,and changes in the state of objects in the scene,the pose estimation is inaccurate,leading to misalignment and drift of the 3D reconstruction model.A three-dimensional reconstruction system based on tightly coupled filtered visual inertial odometry is proposed.This system can be applied to handheld devices with limited computing resources.The multi-state constrained Kalman filter algorithm uses sliding window method for constraint and delay linearization,which can make the data fusion of the 3D reconstruction system based on the multi-state constrained Kalman filter algorithm more accurate.State management deletes redundant information in time,which saves a lot of operating memory.The sliding window filter dynamically changes the sliding window time according to the number of feature points in the window when the computing resources are limited.The old parameters are gradually marginalized,and redundant information is deleted in time,thereby saving a lot of memory.The sliding window working mode can accurately converge,make the inertial information and visual information fuse accurately,and make the pose estimation more accurate,improving the accuracy of pose estimation and the efficiency and effect of 3D reconstruction.The visual inertial odometer based on the multi-state constrained Kalman filter and sliding window filter algorithm is analyzed in detail.Experimental evaluation and comparison are performed on the datasets Starry Night and KITTI.The datasets and large scene experiments prove that the 3D reconstruction system based on handheld image acquisition equipment effectively improves the accuracy of pose estimation,thereby improving the accuracy of 3D reconstruction of large scenesAiming at the shortcomings such as poor immersion,poor display effect and low display efficiency of the 3D reconstruction model,panorama roaming and display for 3D reconstruction system based on the collaborative construction of a PTZ and a handheld image acquisition system is proposed.In this system,the advantages of PTZ and handheld 3D reconstruction system are combined.The PTZ image acquisition system collects the original image information of each single site in a large indoor scene,and provides the processed panorama,the pose of each site,the optimized multi-site pose,the 3D reconstruction model and other information of each site to the 3D reconstruction display system.Combined with the high-precision pose and 3D reconstruction model of the large scene based on the 3D reconstruction system of the handheld image acquisition device,panorama roaming and display for 3D reconstruction system is collaboratively constructed Using technologies such as Three.js and Nginx,the display system can be viewed on a web page.When browsing,the user can enter the interior of the 3D reconstructed model scene and watch the 3D reconstructed model immersively.You can use the cursor to switch the scene for panoramic navigation,and the 3D reconstruction model you browsed has higher definition.By moving the cursor,you can also feel the shape of the objects inside the 3D reconstructed scene and the distance from the viewer by the changed size and angle of the cursor.Finally,through lots of experiments,the accuracy of the 3D reconstruction model was verified.Compared with the existing 3D reconstruction systems,the stability and advancedness of the 3D reconstruction system proposed in this paper can be proved through experiments and results analysis on 3D reconstruction accuracy,efficiency and effect.In this paper,a three-dimensional reconstruction method for large scenes based on PTZ and handheld image acquisition systems is studied in depth,and a self-developed low-cost PTZ image acquisition device and handheld image acquisition device are proposed to establish a highly efficient image acquisition system for Accurate 3D reconstruction.After analysis and comparison of lots of experimental results in different scenarios,the 3D reconstruction method proposed in this paper effectively improves the efficiency of the 3D reconstruction image acquisition system,reduces the cost of the 3D reconstruction image acquisition equipment,improves the performance of the 3D reconstruction system,and optimizes the effect of the 3D reconstruction model.The method proposed in this paper can be used in the fields of virtual reality,augmented reality and robot.