Research On Environment Perception Method Of Multi-source Heterogeneous Sensors For Autonomous Vehicle

Intelligence is the strategic direction of the development of the global automobile industry,and autonomous driving is becoming the focus of competition among countries.Environment perception is the primary link to realize automatic driving,and the basis of vehicle decision-making and control.It provides necessary information such as the position,speed and movement behavior of the surrounding multi-target.Due to the inherent limitations of a single sensor,multi-sensor fusion has become an important way to realize reliable sensing in complex environments,which is the frontier technology direction in this field.However,the design principles and physical characteristics of all kinds of sensors are different,and the description methods and manifestations of the environment are different.There are not only great differences in the adaptability of working conditions,but also coupling correspondence.How to realize the full perception and effective fusion of all kinds of sensors is the difficulty.To meet the practical needs of complex environment perception of autonomous driving,this paper carries out research on key technologies such as joint calibration of multi-source heterogeneous sensors,two-dimensional visual object detection,three-dimensional object detection of point cloud,and fusion detection and tracking of multi-source heterogeneous sensors.The main innovative achievements are as follows:(1)A spatio-temporal joint calibration method for multi-source heterogeneous sensors is constructed.Aiming at the problems of inconsistent time reference and different sampling period of heterogeneous sensors,a hardware triggering method based on GPS time signal and a time synchronization method based on Kalman filter prediction were constructed to achieve the joint calibration of multi-source heterogeneous sensors on the time scale.Aiming at the problem of spatial unsynchronization of heterogeneous sensors,this paper proposed a method of multilidar external parameter calibration based on normal vector estimation and feature matching and a method of Li DAR and camera external parameter calibration based on geometric feature constraint,which realized the joint calibration of multi-source heterogeneous sensors on spatial scale.The joint calibration software of laser,vision and millimeter wave was jointly developed with cooperative enterprises,which improved the calibration efficiency by 50% and laid the foundation for multi-sensor fusion sensing.(2)It innovates a two-stage visual multi-object detection method based on double feature extraction network.For visual target detection algorithm under complex scene of the decline of the precision,based on the Res Net network-FPN candidate regions feature extraction mechanism,realize effective extraction,image characteristics of deep VGG-16 network was used to construct regression module classification,classification and regression effectively solved the candidate regions between the characteristics of the interference problem.Meanwhile,through the classification and regression of multi-scale targets in different feature layers,the detection performance of the network in complex scenes is improved.In PASCAL VOC dataset,the detection accuracy of bicycle categories with large differences in visual angles is 3.8% higher than that of mainstream methods.In MS COCO dataset,the detection accuracy of small target hard to detect samples is improved by 5.0%.(3)A point cloud 3D object detection method based on attention-gated graph convolutional network is proposed.Aiming at the problem that the detection accuracy of point cloud algorithm is insufficient for small targets and occluded targets,the attention gating mechanism is introduced to construct the structure of point cloud image,which strengthens the interaction and update of point cloud semantics and feature extraction network,and improves the feature extraction ability of point cloud target.The most discriminative key nodes of the point cloud image are obtained by selecting the best pooling proportion coefficient.A redundant target filtering algorithm based on driving association area is proposed.The point cloud pavement is extracted by the Otsu threshold,and the driving association area is generated by the subset of edge point species,which effectively improves the classification performance of the algorithm.In KITTI dataset,the recognition rate of pedestrian and cyclist categories is 9.1% and 10.4% higher than that of the mainstream algorithm,respectively.The multi-target detection rate of Nu Scenes dataset reaches 91% after driving correlation region optimization.(4)The target layer fusion sensing method of millimeter wave radar and vision camera is established.For in millimeter wave radar and visual camera perception scope and complementary information completeness,was proposed based on markov distance target adaptive fusion mechanism,through the joint probability data association algorithm,to realize the objective of the millimeter wave radar and visual camera output matching fusion,in six kinds of bad weather scenario 5000 random sample image target detection experiments,The number of successful fusion accounts for about 66% of the total sample number,which effectively improves the robustness of the perceptual system.(5)Innovative multi-target tracking algorithm based on multi-source heterogeneous sensor data fusion.The algorithm is based on the distributed fusion structure,the global nearest neighbor algorithm is used to associate the target data of each sensor,and then the unscented Kalman filter is used to realize the multi-target state estimation,and the latest trajectory of the target is obtained.Horizontal and longitudinal fusion tracking experiments in five complex scenes show that the target tracking accuracy is 87% after multi-source heterogeneous sensor data fusion,which verifies that the proposed algorithm can effectively give full play to the advantages of multi-source heterogeneous sensor data fusion and accurately track multiple moving targets.The proposed algorithm is transplanted and applied based on the real vehicle test platform of the research group.After 12 typical traffic scenarios,the multi-objective sensing accuracy reaches 97%,which can provide reliable environmental information for autonomous vehicles.The research results have important academic and engineering application value.