Research On AUV Intelligent Control Method Based On TD3

In recent years,Autonomous Underwater Vehicle(AUV)has attracted the attention of researchers with its strong autonomy and flexibility when operating on the seabed.Compared with the cabled remote-controlled underwater robot,the AUV is not restricted by the mother ship,and has a larger range of motion and greater flexibility.At the same time,because there is no physical connection with the mother ship,its autonomous control performance is more demanding.Therefore,in order to perform high-precision tasks in a complex marine environment,it is necessary to develop an AUV control algorithm with higher accuracy and robustness.However,when the AUV is performing its mission,it may be disturbed by the surrounding environment including ocean current disturbances and changes in the AUV's own buoyancy at any time.However,most model-based control methods need to preset control parameters in advance.When AUV completes autonomous control tasks such as path tracking underwater,the accuracy and stability are challenged.Based on the control method of Deep Reinforcement Learning(DRL)theory,the agent has undergone a lot of exploration and training,and has excellent autonomous decision-making and anti-interference ability in the face of complex external environments.Therefore,this thesis proposes some improvement schemes based on the Twin Delayed Deep Deterministic policy gradient algorithm(TD3)to study the intelligent control of AUV.The specific research content includes:First,in order to carry out the AUV intelligent control and path tracking experiments,and train an effective reinforcement learning control algorithm model,this thesis first established two reference coordinate systems and established the AUV mathematical model based on the Newton-Euler equation.In addition,in order to test the control performance of the AUV controller designed in this thesis under interference conditions,this thesis adds an ocean current interference model to the AUV operating simulation environment.Secondly,in order to analyze the powerful autonomous decision-making and anti-interference ability of DRL algorithm in robot intelligent control,this thesis introduces the basic theory of reinforcement learning,and analyzes the advantages and disadvantages of model-based and DRL-based control algorithms in AUV control performance.Next,in view of the slow training speed in the reinforcement learning algorithm and the strong randomness in the initial training period,it is difficult to find an excellent strategy.This thesis makes improvements based on the TD3 algorithm,including combining the S-plane control method to complete guided exploration and adding network pre-training stages,etc..In addition,in order to improve the autonomy and anti-interference ability of the AUV in the complex marine environment,this thesis uses the TD3 algorithm to complete the training of the AUV intelligent control task,and adds Gaussian noise to the state during the training process to improve the stability of the AUV controller Sex.Then,based on the improved TD3 algorithm,a reinforcement learning controller was designed,an experimental simulation platform was established,and AUV constant depth control and speed control experiments were designed to verify the superiority of the improved TD3 control algorithm.Finally,based on the improved TD3 algorithm,this thesis uses the line-of-sight(LOS)method to carry out simulation experiments for the AUV linear and curved path tracking tasks.It verifies the advantages of the improved TD3 algorithm proposed in this thesis in training speed and its stability under interference conditions.