Multi-Modal Sensing Fusion Based Accurate 3D Reconstruction Of Natural Environment

Real time and accurate 3D real-time reconstruction of unstructured natural environment is the core problem of robot application scene extending from urbanization environment to field environment.However,the current research faces some problems to be solved,such as low system accuracy,poor reconstruction effect and weak real-time performance.In this paper,a set of multimodal fusion sensing system mounted on mobile flexible scanning robot is designed and developed,and the theoretical research on accurate three-dimensional reconstruction of natural environment is carried out.The main contents include:1.Modeling and calibration of multimodal system based on simultaneous estimation of external participating phantom structure.In this paper,a multi-modal fusion sensing system for field natural environment scanning and reconstruction is designed,which combines the long-range system with the close-range system to complete the unification of the multi-modal coordinate system.The joint observation model of the fusion system and the perceived environment is established,and the system is calibrated by using the method of simultaneous estimation of external participation phantom structure.Through the error analysis and comparison between the experiment and the original method,it is verified that the external participation model structure estimation method has high accuracy and strong practicability.The accurate external parameters obtained by calibration provide accurate system parameters for data registration and three-dimensional reconstruction.2.Multi-scale adaptive natural environment reconstruction based on point cloud and image fusion.The multimodal system integrates a lidar module with remote sparse point cloud accurate ranging and a depth camera with short-range high-precision speckle imaging,so it can be compatible with the function of long-range and closerange 3D reconstruction in natural environment.In this paper,a multi-scale adaptive3 D natural environment reconstruction method based on point cloud and image fusion is proposed.The scale adaptive switching strategy takes the depth threshold as the panoramic scanning index.For the perspective reconstruction,the laser-assisted image reconstruction is completed by setting the threshold for the point cloud feature points and rejecting the external points in the corresponding image feature points.Experiments show that the proposed strategy can better meet the needs of panoramic reconstruction in natural scenes.3.The reconstruction cavity repair method based on DFNN and the real scene reconstruction effect optimization method based on vision inertial measurement fusion motion correction solve the hole problem of point cloud and image fusion real scene reconstruction.The generator network model composed of encoder and decoder is designed.The discriminator network model and the generated network model are mirror images of each other.In addition,aiming at the problems that the multi view 3D reconstruction method of the image is easy to cause the hole of the real scene reconstruction result and the cross stratification of the reconstruction effect caused by no translation increment under the complexity of the scene(such as the corner area),the motion estimation of the system is corrected by jointly minimizing the pre integration constraint of the inertial measurement unit and the visual re projection error in the sliding window.Finally,experiments are carried out on the two proposed methods,and the accuracy and integrity are analyzed to verify the effectiveness and practicability of the proposed meth.