Research On 3D Environment Mapping And Regional Semantic Learning Based On The Fusion Of Camera And Lidar

In the large-scale and complex operating environment,traditional surveying and mapping methods based on 3D laser scanners have problems such as time-consuming and lack of regional semantic information.Aiming at the operating environment with complex structure,large scene span,and light changes,a mobile multi-session scanning and mapping system that integrates cameras and lidar sensors is researched and a regional semantic learning method based on scene recognition is designed.Based on the above methods,a software for dense mapping and point cloud semantic annotation applications is developed.First,aiming at the lack of high-quality data in conventional maps,the joint calibration and data fusion of camera and lidar enables the color information of image to be integrated into the original point cloud data of lidar.In view of the complex and diverse indoor environment and large-scale work area,a multi-session dense mapping method based on sub-maps is proposed.On the basis of Le GO-LOAM,an independent data storage module is designed to realize the storage of dense local map data and an image-based loop detection method is used to assist in generating overlapping point cloud areas between different sub-maps.Then a point cloud matching algorithm is used to solve the transformation relationship between local maps,realizing the fusion of local map data and the efficient construction of global point cloud map.Aiming at the problem of insufficient description of map information in complex indoor environments,a regional semantic recognition method is proposed,which integrates multiple information sources to comprehensively judge regional categories.Through the convolutional neural network model,the overall scene features and local object-level features of environmental images are extracted separately,and they are used together to identify the scene category;The conditional random field(CRF)is used to model the temporal correlation of historical environmental features,so that the regional category labels can be infered in continuous motion process and the accuracy of regional semantic recognition in complex environments is improved.The ground segmentation algorithm is used to extract the ground part of point cloud frame in the map data,and the corresponding colors are rendered for different area categories,which realizes the intuitive description of semantic information of different areas in the global point cloud map.On the basis of the above research methods,the dense mapping and map data processing application software is designed and developed,the functions of which includes joint calibration of sensors,display and interaction of point cloud,data fusion of image and point cloud,dense mapping and storage of map data,map splicing,regional semantic labeling and so on.Through the display window and log window,the software can monitor the process of map data.Finally,the modules and entire system are tested on the real indoor environment dataset,and experimental results verify the effectiveness of proposed methods.