Research On Multimodal Sensor Fusion Technology

Information fusion collects the multi-source information of the target,and then processes,correlates,filters and fuses the data to form the final decision result.In recent years,with the continuous improvement of sensor performance and the application of various deep learning algorithms,information fusion technology has gradually changed from relying on homogeneous sensors to relying on heterogeneous sensors.This multimodal sensor fusion technology can effectively improve the adaptability of the system to the environment,the accuracy of decision-making and the safety of the system.This thesis aims to improve the target recognition rate and tracking accuracy of the fusion system.The main research contents are as follows:1.Aiming at the problem of inconsistent characteristics in the fusion of millimeter wave radar and camera,a fusion system based on point cloud is designed in this thesis.The system can make up for the defects that the camera can not recognize targets in special scenes and the classification effect of millimeter wave radar is poor,and improve the target detection rate.The traditional monocular depth estimation algorithm is also improved.The experimental results show that when the original monocular depth estimation algorithm forms an angle between the target and the sensor,there is a large error in the distance calculation;The distance-pixel based algorithm proposed in this thesis can not only reduce the longitudinal distance error,but also calculate the transverse distance.2.Aiming at the problem of feature fusion of multiple radars,this thesis proposes a fusion algorithm based on dual channel depth neural network,and compares the target recognition effect based on single channel feature extraction network.The simulation results show that the recognition rate based on the dual channel fusion network is higher than that based on the single channel method.3.Aiming at the problem of large tracking error in the fusion of long-range detection radar and infrared sensor,a data level fusion system is built in this thesis.The system uses interactive multi model combined with federated filter to improve target tracking effect.Meanwhile,an algorithm based on unequal interval filtering is used for time synchronization.Compared with the traditional time synchronization method,the algorithm reduces the tracking error.The fusion system is compared with a single sensor.The experimental results show that the tracking effect after fusion is better than that of a single sensor under different signal-to-noise ratios.This thesis further expands the application of multimodal sensor fusion technology in automatic driving,which can provide a new technical scheme for target recognition and target tracking in complex road scenes.