Research On The Motion Control Method Of AUV Based On Reinforcement Learning

Autonomous Underwater Vehicle(AUV),as an important technical means for exploring the ocean,has revolutionary applications in many scientific fields such as marine science,biology,archeology and so on.And its research and development have attracted widespread attention from all walks of life.The complex dynamic characteristics of AUV and the changing marine environment pose great challenges to its motion control.Therefore,it is of great practical significance to study the motion control methods of AUV.Based on an artificial intelligence project,and focusing on AUV intelligent control,this paper introduces reinforcement learning technology into the control system to improve the autonomous learning ability and environmental adaptability of the AUV control system.The main research contents of the paper are as follows:First of all,aiming at the problem that the parameters of AUV model are difficult to be obtained accurately and the environmental adaptability of traditional control methods is poor,a model-free AUV control method based on improved Q-learning is proposed to improve the adaptive ability of the controller onthe basis of improving the self-learning ability of the control system.The speed and heading controllers are designed respectively,and the division of their state,action space and the definition of reward and punishment function are described in detail.This method does not need accurate model parameters,and the optimal mapping relationship between continuous input state and continuous output action can be found independently through the learning mechanism of Q-learning.Compared with the traditional Q-learning method,the introduction of neural network and empirical sample pool enables this method to input and output continuous states and actions,which not only improves the control quality,but also greatly improves the learning efficiency.Simulationresults showthat the proposedmodelfree AUV control method based on improved Q-learning has good self-learning and adaptive ability,and can directlyreplace the traditional controller to control AUV.Secondly,aiming at the problem that the parameters of the controller can not be optimally tuned in practical application and it is difficult to set manually,borrowing that idea of adaptive control and combining the improved Q-learning method with the backstepping controller,a parameter adaptive backstepping control method based on improved Q-learning is put forward,and the design of parameter adaptive backstepping speed and heading controller based on improved Q-learning is described in detail.The self-learning and self-adaptive ability of the parameter adaptive backstepping control method based on improved Q-learning were verified under the conditions of external interference and no external interference,respectively.The experimental results show that the parameter adaptive mechanism based on improved Qlearning can effectively adj ust the parameters of the backstepping controller inreal time through continuous interaction with the environment,and has good adaptive ability.Finally,from the perspective of practical engineering application,the AUV semi-physical simulation system is developed,which takes the real equipment as the test object and fully simulates the operation process of the AUV control system under the actual hardware equipment.At the same time,combined with the planning system,the effectiveness and applicability of model-free AUV control method based on improved Q-learning and the parameter adaptive backstepping control method based on improved Q-learning are further verified through semi-physical simulation system and the reliability of semi-physical simulation system is also verified.