Research On Motion Control Of Unmanned Surface Vehicle Based On Deep Reinforcement Learning

With the continuous development of the world economy and technology,the exploitation and utilization of the ocean are gradually increasing.Unmanned Surface Vehicle(USV),as one of the tools for ocean development and utilization,has been paid more and more attention with its advantages of low cost,high intelligence and strong expansibility.Among the many research fields of USV,motion control is one of the most concerned research fields.In recent years,artificial intelligence technology(AI)has achieved unprecedented development,in which Deep Reinforcement Learning(DRL)framework as the core of the algorithms has caused a great response in the fields of scientific research.More and more researchers are getting involved.Therefore,from the perspective of DRL,this paper studies the motion control problem of USVs.The specific research work is as follows:First,the modeling and control of underactuated USV under a single external disturbance.A modeling algorithm based on Deep Learning(DL)and a control algorithm based on Model Predictive Control(MPC)are designed.In the modeling stage,the Deep Feedforward Neural Network(DNN)is trained by collecting the off-line state-action data of the underactuated USV as the training samples and combining with the Adam optimization method to make it fit the dynamics model of the USV.In the control stage,the DNN established by DL algorithm is used as the prediction model and the MPC algorithm is used to select the optimal control strategies.Finally,the feasibility and effectiveness of the algorithm are verified by simulating the task of tracking the desired path and trajectory.Second,a control algorithm based on DRL is designed to solve the problems of multiple disturbances in the time-varying Marine environment,too much training data used by DL algorithm and the possibility that the last dynamics model DNN for offline training cannot adapt to the new environment.Firstly,a small amount of offline data is used to train the DNN,and then combined with MPC algorithm to control the USV and collected online state-action data of the USV.Then using the new and old data together to strengthen the training the last DNN,through constant alternation of collecting data and intensive training,until the setting maximum number of iterations is reached.Finally,the effectiveness of the algorithm is verified by the simulation of tracking the desired path and trajectory.Third,considering the characteristics of strong coupling,constraint and weak anti-jamming capabilities of the underactuated USV,multi-disturbance in the time-varying marine environment and the influence of DRL algorithm on the control effect under different parameters,a control algorithm based on DRL is adopted to the control problem of underactuated USV.By collecting online data alternately and strengthening training DNN,the DNN is continuously optimized until the maximum number of iterations is reached.Finally,the effectiveness of the algorithm is verified by simulating the desired path and trajectory of the USV and comparing the control effect of the DRL algorithm under different parameters.