Research On Joint Ship Detection And Water Surface Semantic Segmentation Method For Inland River Environmental Perception Of Unmanned Surface Vehicles

For autonomous inland navigation,Unmanned Surface Vehicles(USVs)sense the surrounding environment information in real-time by optical or other types of sensors,and plan and make decisions to achieve obstacle avoidance and collision avoidance behaviors.Due to the characteristics of low energy consumption,wide field of view and high perception capability,the optical camera-based environmental perception method has become the main technical approach in the field of environmental perception for USVs.For the visual environment perception problem of inland USVs,this thesis proposes a new joint object detection and semantic segmentation framework based on hybrid attention mechanism,which can detect inland ship objects and water surface semantic segmentation simultaneously and can improve the efficiency and reliability of inland river visual environmental perception.In addition,the method proposed in this thesis is characterized by a low number of parameters and low computational effort,which can be deployed on an embedded edge computing platform and reduces the development cost of small autonomous inland river USVs.The main research components are as follows:(1)In response to the lack of datasets in the study of visual environment perception of inland river USVs,a dataset that can be used in the study of object detection and water surface semantic segmentation of inland river ships is constructed and published.Mainly,inland waterways are captured in the field,and the image data are manually cleaned and labeled for ship detection and semantic segmentation.In addition,by incorporating other public datasets,more channel environments are covered to promote the research progress of visual environmental perception of inland river USVs.(2)A joint ship detection and water surface semantic segmentation convolutional neural network framework based on attention mechanism is proposed to address the problem of environmental perception implementation methods for inland river USVs.It is capable of parallel real-time inference of inland river ship object detection and water surface semantic segmentation tasks(segmentation into water surface,ship,and background),including shared feature extraction backbone,object detection branch,and semantic segmentation branch.Compared with separate ship object detection networks or water surface semantic segmentation networks,the proposed method greatly reduces the computational cost while ensuring the visual perception of inland river USVs,and helps to deploy on embedded devices.(3)A new hybrid attention mechanism is introduced to achieve multi-level feature information fusion between the detection branch of the object and the semantic segmentation branch of the semantic segmentation,in view of the narrow inland waterways and congested ship traffic,and combining the correlation between the ship object detection and the semantic segmentation task on the water surface in visual perception.The joint network implemented based on this method has higher accuracy of ship detection and water surface segmentation.(4)The proposed network model is trained and validated based on the inland waterway dataset constructed in this thesis,and further comparative validation experiments are done based on other partially public datasets.The results show the effectiveness,robustness and generalization ability of the proposed joint ship detection and water surface semantic segmentation model based on the attention mechanism for inland waterway visual environmental perception applications.