Research On Multi-Path Following For Unmanned Surface Vehicle Based On Deep Reinforcement Learning

As a key technology in the motion control of Unmanned Surface Vehicles(USVs),path following control is an important foundation for the autonomous navigation quality of USVs and has long been a research hotspot in the field.Existing path following methods have mainly focused on single fixed-path following tasks,and the following performance cannot be guaranteed when the task path changes.In recent years,with the rapid development of artificial intelligence technology,an increasing number of studies have begun to apply it to USV path tracking research.Among them,deep reinforcement learning(DRL)methods have powerful representation and decision-making capabilities and have shown great potential in path tracking control.To address this issue,this thesis proposes the G-SAC algorithm based on deep reinforcement learning to achieve multi-path following control for USVs.Furthermore,considering the complexity of the USV operating environment on the water surface,the proposed method is improved,and further introduce the LSTM-GSAC algorithm to achieve multi-path following control of USVs in obstacle environments.The specific work and achievements of this thesis are as follows:To construct a suitable USV model to complete the training,validation,and evaluation of control algorithms,this thesis establishes the USV motion reference frame,provides a description of the six degrees of freedom force and motion of USV,and constructs a six-degreeof-freedom manipulation model of USV that includes disturbing factors.Based on this,from the perspective of USV multi-path following control,the model is simplified to obtain a threedegree-of-freedom nonlinear manipulation model of USV.To verify the accuracy of the model,simulation examples were designed based on actual ship data,laying a foundation for subsequent research on multi-path following control algorithms.To effectively solve the problem of multi-path following for USVs and improve the reuse rate of trajectory experience between different tasks during algorithm training,this thesis proposes the G-SAC path following control algorithm based on the generalized posterior experience replay mechanism,and models the multi-task Markov decision process for USV multi-path following problems.By matching the current trajectory with a more suitable task and storing them together in the experience pool through re-labeling of the USV running trajectory,the reuse of trajectory data between different tasks is effectively enhanced.Simulation results demonstrate that the USV controller based on G-SAC has good training efficiency and optimal following performance in multi-path following tasks.To ensure that USVs can still complete multi-path following tasks in obstacle environments while also considering autonomous obstacle avoidance tasks,this thesis introduces LSTM network to improve the G-SAC algorithm and propose the LSTM-GSAC algorithm.The LSTM-GSAC algorithm extracts and processes the temporal information of USV operation,enabling the USV to have a certain degree of autonomous obstacle avoidance capability.Simulation results show that the USV controller based on the LSTM-GSAC algorithm can accurately follow the predetermined path and avoid obstacles in multi-path following tasks in obstacle environments,demonstrating higher success rates and better following performance.To meet the simulation functional requirements and design specifications for USV multipath following,the intelligent control simulation software for USVs was designed and developed.It provides functions such as USV model management,USV control task simulation,real-time simulation demonstration,USV status detection,and control algorithm model training,targeting both operators and researchers,demonstrating the effectiveness of the software during testing.