Unmanned Target Detection Based On Millimeter Wave Radar And Vision Fusion

Unmanned target detection is an important development direction.Due to the diversity of driving environment,it is often impossible to accurately obtain information such as category,location,speed,etc.During target detection,using only a single sensor to detect often does not accurately obtain information about the category,location,speed,etc.of the target being checked at the same time.In response to this problem,this paper takes millimeter-wave radar and camera as the research object,combines deep learning and related theoretical methods,builds a deep learning target detection network based on the fusion of mmwave radar and visual fusion of multi-data sources,and solves the problem of unmanned forward target detection in a network-level fusion way.In this thesis,based on the millimeter wave radar data and camera image data transformation,the coordinate space of the two sensors is unified based on the residual BP neural network transformation algorithm.This paper analyzes the coordinate transformation mode,the minimum dip transformation mode and the traditional BP neural network transformation mode,and puts forward the improvement scheme in the traditional BP neural network transformation mode.The introduction of one-dimensional convolution to replace part of the full connection of traditional BP neural networks enhances the expression ability of the network and reduces the number of parameters in the network,and the design of residual structure increases the shallow characteristic information of the deep network and improves the accuracy of the model.The residual BP neural network transformation algorithm has been tested to provide effective data transformation with an average predictive error of 13.57 pixels and a test speed of only 0.36 ms on driverless platforms.In view of the problem of data synchronization,the optimization scheme of near-order pumping frame is proposed,which makes the time difference between the synchronous sampling time of the mmwave radar and the camera double that of the pre-sequence pumping frame.The Deep Learning Target Detection Network(MS-YOLO)based on the fusion of millimeter-wave radar and vision multi-data sources is designed.MS-YOLO builds a dual backbone based on the CSPDark Net backbone with millimeter-wave radar and image data characteristics,extracts the pre-characteristics of the two,and fuses the two features at the detection layer,uses the Foncs structure to slice the inputs and samples them,introduces the CSP structure to enhance network learning capabilities and reduce the number of parameters,and builds the SPP,FPN,PAN three multi-scale structure for the network to introduce multiscale characteristics.A joint dataset MS-Dataset containing 7000 sets of millimeter-wave radars and images was produced for the training fusion network.By training and testing the MS-YOLO network in the MS-Dataset dataset,the m AP of the MS-YOLO network increased by 9.6 points to 84.1 and the frame rate reached 65 fps.In this thesis,the unmanned target detection system of mmwave radar and visual fusion is realized by combining the residual BP neural network with the MS-YOLO network for data transformation.Experiments show that the system can provide real-time forward target information detection for unmanned systems.