Research On AUV Path Planning And Tracking Methods Based On SAC Algorithm

In recent years,autonomous underwater vehicle(AUV)has attracted increasing attention from researchers due to its strong flexibility and autonomy in underwater operations.AUV technology has been increasingly applied in various fields such as marine resource exploration,marine environmental monitoring,maritime rescue,and marine defense.AUVs have the advantages of good flexibility and wide range of activities due to being not constrained by the mother ship.However,the underwater environment is complex and variable,and AUVs are subject to various unpredictable external disturbances during operation.In addition,modern underwater tasks have increasingly high requirements for the real-time,accuracy,robustness,and other aspects of AUV control algorithms.Therefore,for good real-time performance,high accuracy The research on AUV control algorithms with strong environmental adaptability is very necessary.This article mainly focuses on the two main control tasks faced by AUVs during underwater operations,namely AUV path planning task and AUV path tracking task.For AUV path planning tasks,the current main planning algorithms such as A*,artificial potential field method,and swarm intelligence algorithm all have their own advantages,but there are also some problems,such as poor real-time performance and adaptability.For path tracking tasks,the main model-based control algorithms currently require setting model parameters in advance,which poses challenges to the stability and adaptability of the algorithms in practical tasks.The algorithm based on deep reinforcement learning(DRL)theory continuously interacts with the environment through intelligent agents and trains repeatedly.The trained model has strong environmental adaptability and anti-interference ability,and can be applied in path planning and path tracking tasks.The soft actor critic(SAC)algorithm has been increasingly used in the field of robot control in recent years due to its strong probing ability and high stability.Therefore,this article conducts experimental research on AUV path planning and AUV path tracking tasks based on the SAC algorithm.The specific research content includes:(1)Designed an AUV path planning experiment in a three-dimensional environment for the AUV path planning task.In response to the slow convergence speed of AUV path planning task training and the tendency to fail quickly in the early stages of training without obtaining effective rewards,an improved SAC algorithm based on the artificial potential field method is proposed.The action selection strategy of the SAC algorithm is improved,and the action calculated by the artificial potential field method is used with a certain probability to achieve guiding exploration during action selection.In addition,in order to verify the algorithm’s antiinterference ability and improve the adaptability of AUVs in complex marine environments,this paper added ocean current interference factors to the simulation environment of AUV path planning tasks,namely interference on the AUV’s own linear velocity and interference from the random movement of obstacles around the AUV.Finally,experiments were conducted in a simulation environment,and the experimental results of the improved SAC algorithm proposed in this paper,original SAC algorithm,DDPG algorithm,and TD3 algorithm were compared to verify the superiority of the proposed SAC algorithm based on artificial potential field method in terms of training convergence speed,training stability,and anti-interference ability.(2)Designed AUV straight line path tracking experiments and AUV curve path tracking experiments for AUV path tracking tasks,and improved the reward function of the SAC algorithm based on the line of sight method.Added rewards for the difference between AUV yaw angle and expected yaw angle,as well as changes in the distance between AUV current position and the line of sight target point,in the reward function,which improved the training convergence speed and stability of the SAC algorithm.This article uses the improved SAC algorithm,original SAC algorithm,DDPG algorithm,and TD3 algorithm for training validation.At the same time,simulation ocean current interference experiments were also conducted for path tracking tasks,verifying the advantages of the proposed SAC algorithm based on the line of sight method in training stability,tracking accuracy,and anti-interference ability.(3)The experiments in this article need to be conducted in a simulated simulation environment.To ensure the smooth progress of path planning and path tracking experiments,this article establishes an AUV mathematical model and a simulated environment map model.For the mathematical model of AUV,two sets of coordinate systems are established,namely geodetic coordinate system and AUV self body coordinate system,and then the kinematics equation and dynamic equation of AUV are established respectively.A 3D map environment for AUV path planning and a 2D map environment for AUV path tracking were established based on the OpenAI Gym framework for simulating environmental maps.