Research On Obstacle Avoidance For AUV Based On Reinforcement Learning

Twenty-first Century is the century of the ocean, the ocean contains a wealth of resources and infinite mysteries to be explored. As an important tool to explore the ocean, Autonomous Underwater Vehicle(AUV) has been paid more and more attention by the researchers. AUV is a high-level underwater robot with intelligent behavior, it has the characteristics of wide range of activities, flexibility, good concealment, and can work in complex marine environment. However,the application of AUV is still facing some challenges. Because of the wide operating range and often bearing the underwater exploration work, AUV often need to work in unknown, complex and difficult to predict under-water environment. So the control of AUV raised a high demand.Among the local path planning methods, how to avoid obstacles and reach the target successfully is one of the important tasks in the research of AUV.In the present study, artificial potential field, artificial intelligence and reinforcement learning methods are the most widely used methods in the research of the obstacle avoidance.Among them, the reinforcement learning method does not require a priori knowledge. In addition,it has a strong self-learning ability. So it is suitable for AUV to avoid obstacles in the unknown environment, and has great potential application in the AUV research.Reinforcement learning belongs to machine learning, it is a very important branch of machine learning. The process of reinforcement learning is the process of probing environment repeatedly, similar to the process of trial and error adopted by animals in the unknown environment. By learning, the animals can obtain an optimal action strategy in the unknown environment so as to obtain the maximum return. Compared with other learning strategies, the biggest advantage of reinforcement learning is that it does not require complete a priori knowledge or even a priori knowledge, but it is still able to guarantee good robustness and adaptability.In this thesis, we study the method of avoiding obstacle on two-dimensional plane based on reinforcement learning in AUV. Firstly, we study the system structure and implement method of reinforcement learning. Then, we study the implementation of input module, output module and strategy module in reinforcement learning. In this thesis, the basic principle, algorithm and characteristics of Q-learning are studied, and we proposed a method to deal with the disadvantages of the slow convergence speed of Q-learning. And the learning efficiency is improved.In the traditional reinforcement learning method, there is a problem of the curse of dimensionality. A method to solve the problem is a generalization of the traditional reinforcement learning method. Based on the research and improvement of Q- learning algorithm,the neural network is applied to the reinforcement learning algorithm, the neural network method is used to solve the curse of dimensionality, and the Q-learning algorithm based CMAC network is proposed, which is used in the research of AUV obstacle avoidance.Finally, this thesis uses AUV to simulate the obstacle avoidance in the two-dimensionalplane. In the experiment, we use the conventional Q-learning algorithm and the improved algorithm proposed in this thesis to carry out the plan. The experimental results verify the effectiveness of the proposed algorithm.