Research On Path Following Control Method Of Unmanned Ship Based On Deep Reinforcement Learning

Highly autonomous and intelligent unmanned ships are the inevitable trend of the development of shipbuilding and shipping industry.As one of the basic tasks of motion control of unmanned ships,path following is the key to realize autonomous and intelligent navigation of unmanned ships.Due to the uncertainty of ship dynamics and the interference of environment,the path following control can be regarded as the optimal control problem of a complex nonlinear system.The study of efficient intelligent control algorithm will help to improve the level of ship intelligence and further lay the foundation for the realization of autonomous navigation.Taking the path following control of underactuated unmanned ships as the starting point,this paper studies the intelligent control strategy of path following of unmanned ships from the perspective of guidance and control.The main research work is summarized as follows:(1)In order to realize the training and evaluation of the ship’s intelligent control strategy,an unmanned ship simulation test platform is constructed for the problems of high risk,long cycle and high cost in the real ship test.The characteristics of ship motion are analyzed,a three degree-of-freedom plane motion model is established,and the simulation control of ship motion is realized by programming in Unity.Through the application program interface,the information exchange between the external environment and the virtual environment is realized.In Py Charm,the manoeuvrability of the ship is tested through the full-turn experiment and the Z-shape experiment.The simulation results show that the ship has good manoeuvrability,and a more realistic motion simulation is realized.(2)An indirect control scheme combining guidance and heading control is proposed for path following control of underactuated unmanned ships.Line of sight(LOS)guidance algorithm is used to obtain the desired heading,PID control method is used to realize heading control,and BP neural network is used to improve the adaptive ability of PID controller.In addition,in order to solve the problems of sudden change of heading and slow convergence speed in LOS algorithm,the ship turning radius and follow-up error are defined as nonlinear variation relationship,and the radius of path point acceptance circle is adjusted adaptively.The effect of this method is verified by simulation on the unmanned ship simulation experimental platform.The results show that the adaptive LOS guidance algorithm combined with BP-PID control has strong adaptive ability,effectively improves the ship control stability and achieves good path following effect.(3)A path following control strategy based on the Deep Deterministic Policy Gradient(DDPG)algorithm is proposed to achieve end-to-end intelligent control for the path following control problem under unknown environmental disturbance and model uncertainty.Aiming at the problem of long training time caused by low data efficiency in DDPG algorithm,an improved DDPG algorithm with online feedback mechanism is proposed by introducing the idea of supervised learning.The experimental results show that the improved DDPG algorithm has stronger learning ability and faster convergence speed.In terms of control performance,it has better control effect than DQN,BP-PID and DDPG.It can automatically adjust the control strategy according to the changes of the current state,and the control process is more stable.This paper studies the path following intelligent control algorithm of unmanned ships from the perspective of autonomous learning,which can meet the needs of path following in unknown environment.The research results are of great value in the autonomous navigation of unmanned ships.