Research On Water Surface Target Perception Method Based On Multi-source Information Fusion

With the development and utilization of marine resources getting more and more attention,the development of marine equipment and technology is also more important.As a typical representative of intelligent marine equipment,Unmanned Surface Vehicles(USV)can realize functions such as autonomous planning and navigation,environment perception,target detection,and autonomous obstacle avoidance,so it has a wide range of application values.Environmental perception is a prerequisite for unmanned systems to effectively complete various established tasks.Target detection is a basic task in the field of environmental perception and computer vision.Improving the accuracy and speed of target detection algorithms is of great significance to improving the working capabilities of various types of unmanned equipment.Visible light cameras,infrared cameras,and lidar are commonly used sensors in current unmanned systems,but the above sensors are limited by imaging principles,and cannot meet the perception needs in complex marine environments when they work alone.In order to better complete the task of surface target perception,this paper adopts the method of multi-source information fusion and combines the characteristics of the three sensors to jointly complete the detection task.The main research work of this paper is as follows:First,a multi-sensor calibration method and fusion scheme are investigated.The research in this paper involves three sensors,namely visible camera,infrared camera and LIDAR.This paper briefly introduces the basic principles and calibration principles of the three sensors.Based on the existing calibration theories and methods,a calibration scheme with the visible camera as the calibration center is developed.Due to the special characteristics of the infrared camera,the conversion relationship between the visible image and the infrared image is determined by the image alignment method.And based on the similar data forms of visible and infrared images,a fusion scheme was developed to fuse visible images with infrared images first and then with Li DAR data for a second time.Secondly,the infrared visible image fusion algorithm is investigated.In this paper,a new unsupervised infrared visible fusion algorithm based on convolutional neural network is proposed.For the characteristics of infrared visible light fusion algorithm with multiple inputs and single outputs.Based on the existing algorithm,this paper improves the image visible light fusion algorithm by improving the feature extraction network and task description method.And the targeted optimization is carried out for the data characteristics of the water surface environment,and the fusion effect is compared with the experiments.The results show that the infrared visible light fusion algorithm proposed in this paper for the water surface environment achieves better results in both subjective comparison and objective indexes.Then,the fusion and detection of images and LIDAR are investigated.According to the characteristics that the surface target is far away and the point cloud collected by Li DAR is more sparse.The filtering link is eliminated to retain more lidar data.A signal-level pre-fusion method is used to project the Li DAR data to the image and fuse it into an image.Then YOLOv4 is used to perform target detection on the fused data.Experiments show that the fused data can accomplish the target detection task.Finally,a time-synchronized three-sensor water surface dataset is constructed in this paper.The effectiveness of the fusion method used in this paper is verified.For the data in different fusion stages,the exact same detection method is used and other conditions are kept constant to conduct comparative experiments on the effect of two fusions.The experimental results show that the multi-source information fusion method proposed in this paper can well improve the water surface target perception capability.