Research On Path Tracking Method Of Underwater Vehicle Based On Reinforcement Learning

Autonomous underwater vehicles(AUV)have become an important equipment for human to develop the ocean.However,complex control structures are required due to the characteristics of the AUV with nonlinear kinetic systems,strong coupling,and parameter uncertainty.An underwater robot can’t exactly navigate the path it’s set on a mission.Therefore,the modeling and pathway tracking of AUV have become an important part of AUV research.The autonomous control that can adapt to the change of Marine environment is the core technology to realize the autonomy of underwater robot.This topic mainly studies how to complete the path tracking task of underwater robot more accurately,and designs the path tracking control method of underwater robot according to the uncertainty of underwater robot parameters and external interference.First of all,in order to complete the path tracking control of underwater,it is necessary to define and transform the coordinate system,Kinematic modeling and dynamic modeling are carried out respectively to obtain underwater six-degree-of-freedom control model.The influence of relevant interference on underwater vehicles is analyzed,and model-related simulation experiments are designed and completed.Secondly,in order to track the target path accurately,an end-to-end Q-learning path tracking control method is designed based on the line of sight navigation method and related reinforcement learning idea.Markov quadrilateral is defined for the underwater vehicle,and state and action space are designed to enhance the learning ability of the controller.Finally,In order to solve the problem of discretization of Q-learning in action space and state space,a gradient reinforcement learning control algorithm based on depth determination strategy of dual neural network model is proposed.The 3D path tracking control experiment of underwater robot is completed,and the tracking performance of the controller based on the improved neural network reinforcement learning algorithm is verified.The results show that the designed 3D path-tracking controller is closer to the target trajectory than the conventional controller and has a higher robustness.