Research On The Fusion Of Heterogeneous Sensors And Its Application In Scene Understanding

Object detection and road detection are two core parts for automatic car,however,it’s hard to provide reliable perception results just employing single sensor.Thus,the proposed methods in this paper focus on the fusion of different sensors.The main contributions and innovations of our work are described as follows:1)We design a Lidar-to-Camera calibration algorithm based on the result of calibration board modeling,and introduce a method to locate targets with the data fusion.This work achieves the precise calibration result between the camera and lidar,and then we can locate the targets in the vehicle coordinates by using DBSCAN clustering algorithm.2)We propose a lanemarkings detection algorithm via the fusion of IMU and camera,which is implemented by the multi-stage hough space calculation and CNNs.A lane fitting method based on the spectral-clustering is used to cluster line segments into different lane instances,and finally we use a geometry model to describe the lanemarkings.3)We design a road segmentation networks based on the fusion of lidar and image.This networks can be used in different kinds of road type such as the asphalt road,oil road and structured road.Detailed ablation experiments prove that the performance of the fusion networks is better than segmentation networks just using image.We perform series experiments on the automatic platforms and the experiment results validate the effectiveness of the works proposed in this paper.