Research On Environmental Perception Method Based On Fusion Of Vision And Lidar Information

In recent years,with the rapid development of deep learning theories and methods,it has become possible for wearable exoskeleton systems to perceive complex environmental scenes using neural networks.The task of environmental perception requires the analysis of data returned from various sensors,of which object detection is an important prerequisite for environmental perception.However,for complex scene environments such as streetscape traffic environment,it is difficult to meet the 3D perception requirements using data obtained from low-cost single-line Light Detection and Ranging(Li DAR).While the existing multi-line Li DAR detection algorithms do not make full use of the spatial structure information of 3D point clouds,and the detection of long-range objects often leads to missed and false detection due to the spatial sparsity of points.To address the above problems,this thesis utilizes monocular camera and multi-line Li DAR to acquire visible images of the environment and point cloud data respectively,and studies the object detection method based on point cloud data and the method fused image and point cloud data,providing parametric sensing of targets for the surrounding scene.the main contents of this thesis are as follows:(1)Object detection algorithm based on point cloud data: Since single-line Li DAR can only express the planar information in the environment,multi-line Li DAR is selected for spatial information perception.For the irregular characteristics of point cloud data,the regular convolution in image processing is no longer applicable,so the graph convolution operator is used to process point cloud data more flexibly.A multi-scale graph structure based on spatial grid down-sampling is designed to characterize the point cloud data;an alignment offset is introduced in the graph convolution operator to compensate for the cumulative variance problem caused by the small displacement of local neighborhood vertices.At the same time,in order to utilize useful information more effectively in the edge feature aggregation network and alleviate the influence of noise on the model,an aggregation method based on attention mechanism is adopted to complete the feature extraction of point cloud data in the graph convolution network;finally,a soft non-maximal suppression algorithm is used to obtain the final regression object frame and its category scores.The test results on the KITTI public dataset show that the proposed algorithm achieves competitive results in the object detection of vehicle categories.(2)An object detection algorithm by fusing image semantic information and point cloud data: To address the problem of sparse point cloud data under weak and distant objects,a method of fusing visible image semantic segmentation results with point cloud data is proposed to improve the detection accuracy of small objects.First,the pre-trained image semantic segmentation model is used to fine-tune on different data sets to obtain pixel-level category labels for image semantic segmentation,and then projected to the point cloud space through the projection matrix.Finally,the point cloud frame integrated with image semantic information is input into the point cloud detector.The fusion semantic model is evaluated by selecting detectors with different point cloud representation and feature extraction methods.Experiments on the KITTI dataset show that the addition of semantic information in all types of detectors improves the accuracy of the models to some extent,especially in the detection of small objects at long distances such as pedestrians,where the image data is more helpful in improving the object detection results due to its higher resolution.In summary,the environment perception method fusing vision and Li DAR information proposed in this thesis can fully exploit the spatially structured information of 3D point clouds,and solve the problem of decreasing detection accuracy of small objects to a certain extent by fusing visual semantic information in weak and long-range object scenarios.It provides intelligent environmental perception assistance for wearable exoskeleton equipment.