Mobile Robot Path Planning Based On Hierarchical Reinforcement Learning

With the rapid development of science and technology, mobile robot has been received more and more application. Path planning is one of the most fundamental and most important tasks in mobile robots. And it shows the robot's exchanging ability with around environment. So the mobile robot's path planning has important and practical meaning. HRL is researched in this thesis. Path planning problem for mobile robots under dynamic environments is comprehensively studied to deal with the problem. The main work lists as follows:(1) Common methods of path planning are analysed in this thesis. The theory of reinforcement learning is studied and the theory of hierarchical reinforcement learning(HRL) is realized.(2) The thesis proposes a layered framework for HRL. The system of the path planning has been divided into three layers from high-level to low-level. The lower layer provides services for the upper layer, and is transparent for the upper, thus, the framework is easy to expand. Machine learning techniques are used in each layer and the higher layer is implemented baced on lower layer. At the same time, the technique can overcome the limitation of hand-coded implementation.(3) The basic principle and the algorithm of Q-learning are studied. Aiming at the slow convergent rate of Q-learning, CMAC, as a local generalization neual network, is adopted to develop the algorithm of Q-learning. And credit assignments of the data, which have being learned,must be fully considered when adjusting the weight of networks. So it obviously improves online learning speed and its accuracy of conventional CMAC. Local path planning in complex environments are completed and the effects are good. Then, this thesis implements the algorithm of MAXQ which based on CMAC. For mobile robots, it is very suitable to dynamic real time on-line control.(4) The obstacles in complex environment are divided into three kinds: protruding static obstacles, concave static obstacles and moveable obstacles which have different directions. Aiming at different kinds of obstacles, this thesis gives the methods of avoidance and resolves local path planning of mobile robots based on HRL in the environment with complex obstacles.VC++ is used to program software in the computer, and the simulation results of several kinds of algorithms demonstrate the efficiency of hierarchical reinforcement learning. The computer experiments show it is fit for solving path planning problem under complex environment.