Research On Unmanned Vehicle To Obstacle Detection In Open-Pit Mines Based On Cross-Modal Fusion

With the implementation of national green mine and mine intelligent construction specifications,driverless mine truck technology has been implemented in the production and transportation of open pit mines,and the real-time intelligent perception of mine road environment is one of the keys to the safe operation of open pit mine transportation trucks as the hub of interaction between driverless mine trucks and external environmental information.As the "eyes" of the unmanned vehicle,environmental intelligent perception provides important information support for the truck’s decision-making system.The fusion of image recognition and LIDAR perception method has become a hot spot for obstacle detection due to its high recognition rate and accurate distance measurement.In this paper,we focus on obstacle detection based on deep learning and cross-modal data fusion methods,and conduct an in-depth study on unmanned truck travel obstacle detection in open pit mines.The main contents of the research in this paper include:(1)Vehicle obstacle image pre-processing and dataset construction study.Through field acquisition and manual labeling,road obstacles such as gravel and potholes in open pit areas were added,while the vehicle types driving in the road were further subdivided.In order to solve the problem of uneven sample distribution in the dataset during training,traditional data augmentation methods and Poisson fusion-based small target enhancement algorithms are used to further enhance the data features of small targets and multi-scale obstacles.(2)Research on machine vision-based obstacle detection model for mining roads.Aiming at the complex and variable unstructured roads in open pit mining areas,current detection methods cannot meet the multi-scale and small-target detection capability of obstacles,Rep VGG+ backbone network,which is more adapted to target detection,is proposed in the feature extraction stage;in the feature fusion stage,a bidirectional feature fusion pyramid model based on Sim AM space with channel attention and cross-stage connectivity is proposed.The performance of small target obstacle detection is improved by expanding the feature map and feature perceptual field for predicting small target obstacles,and the multi-scale detection performance is improved by the bidirectional feature fusion mechanism.The network classification prediction module and loss function are also further optimized to improve the obstacle detection performance,reduce feature redundancy and speed up model inference,thus realizing real-time accurate detection of road obstacles in mining areas.(3)A study of unmanned mining truck front obstacle determination method based on cross-modal data fusion.By calculating the internal reference of the camera,the image distortion is corrected and the exact position of the pixel in the image coordinate system is obtained.And the outer reference between image coordinate system and Li DAR is calculated using the feature point matching method to obtain the conversion method between two coordinate systems to achieve cross-modal fusion of image and point cloud data.The non-central suppression algorithm is also proposed in obstacle spatial position localization to achieve the accurate measurement of the distance of the traveling obstacle target in the open pit mine area.Through the analysis of experimental results,it can be seen that the fusion detection method of unmanned vehicle traveling in open pit mine proposed in this paper has good detection effect on multi-scale small target obstacles in the road of open pit mine,and at the same time,in the distance detection of road obstacles,it has higher accuracy compared with the machine vision method,and can complete accurate determination of obstacles in front of the mine card.Finally,the method of this paper is applied to the unmanned mine truck in the open pit mine for experimental testing,and the results show that the method can meet the requirements of accurate detection of unstructured road obstacles in the open pit mine area.