| With the development of science and technology,the fields of automatic driving,intelligent transportation and robot are rising rapidly in this context.People have higher and higher requirements on quality,timeliness and dimension of environmental perception.Using two-dimensional image alone for perception can no longer meet the requirements.Three-dimensional point cloud comes into being.The fusion registration between two-dimensional image data and three-dimensional point cloud data plays a crucial role in the field of AI research.For example,traffic monitoring images can be registered with the surrounding point cloud to improve the accuracy and robustness of traffic environment perception.However,due to the complexity of the environment,the unknown internal parameters of the monitoring camera and the inconvenient calibration,the lack of cross-modal training data,the diversification of point cloud data,and a variety of noise interference,the image registration with the surrounding color point cloud is still faced with great challenges.On the one hand,the traditional image point cloud registration is to manually select the2D-3D matching key points,and obtain the external parameters of the camera through Pn P method on the basis of known camera internal parameters.This method will consume a lot of manpower,material resources and time cost;On the other hand,in the existing researches,registration methods are mostly based on the same mode,and the research on registration under different modes is just emerging.In recent years,deep learning has developed rapidly.Its high-dimensional and robust feature coding and feature transfer capabilities make deep learning widely applied in the field of AI.Based on deep learning,cross-modal registration of image and point cloud is studied in this thesis.The main research contents are as follows:(1)An image-color point cloud cross-mode registration method based on graph neural network is proposed.Firstly,the internal and external parameters of the camera are initialized,and the color point cloud is projected into a color image by using the internal and external parameters of the camera,and the 3D-2D mapping relationship is obtained.Then,the matching2D-2D key point pairs between the camera image and the projected image are automatically obtained by the graph neural network.Finally,the matching 2D-3D key point pairs between the camera image and the color point cloud are obtained by mapping transmission.Finally,alternate direction multiplier method(ADMM)is used to optimize the internal and external parameters of the camera.In this registration method,a method is proposed to initialize the camera parameters inside and outside the point cloud,which solves the difficult problem of camera parameter initialization in 10-DOF space.Based on the graph neural network,the difficulty of 2D-3D key point matching is solved by mapping transmission.The alternate direction multiplier(ADMM)algorithm is used to optimize the internal and external parameters of the camera at the same time when the internal parameters of the camera are unknown.This registration method improves the universality and registration efficiency of the algorithm,and provides a basis for the large-scale application of image-color point cloud data fusion in intelligent transportation and other fields.(2)An image-color point cloud cross-modal registration method based on iterative reprojection is proposed.The proposed method further optimizes the above method and improves the accuracy of cross-mode registration.This method is based on iterative reprojection to solve the problem of inaccurate initial camera parameters and transmission of point cloud in the process of projection to the image,resulting in 3D-2D mapping errors.In the process of obtaining 2D-2D key point pairs based on graph neural network,only local features but not global features are considered,which leads to the mismatching of 2D-2D key points.In summary,the iterative reprojection method is adopted to reduce the projection loss from point cloud to image,and K-means clustering method is used to eliminate the mismatching of 2D-2D key points based on local features from a global perspective,thus improving the accuracy of registration.(3)Based on the above two algorithms,this thesis designs and implements the image-point cloud cross-modal registration system.On the basis of fully considering various traffic dynamics and cross-modal registration requirements,the system predefined three modules,namely,data processing module,two-dimensional and three-dimensional registration module and registration optimization module.Users can obtain accurate 2D-3D registration results in the case of unknown camera parameters through this system,which provides convenience for subsequent3 D reconstruction,environment perception and other work. |
