Multi Agent Path Planning And Formation Based On Hierarchical Reinforcement Learning

Now the research of multi agent system is the most advanced direction in the field of artificial intelligence and automation control.It has a great application in all walks of life,this is due to their own distribution,large redundancy and robustness,as well as good cooperation and adaptability and other advantages which a single agent system does not have.In the actual situation,multi agent were generally working in unknown dynamic environment,which all kinds of dynamic and static obstacles couldn't be known by agent,Therefore,it needs to have a strong sense of the environment and adaptability,when dealing with these unexpected situations.Agent had the ability of self-learning and online learning when the reinforcement learning ability on a no environmental model of learning.And this phenomenon has been paid more and more attention by researchers.But the biggest flaw of reinforcement learning was the problem of "dimension disaster" when it came to a complex task.The learning task was divided into different sub tasks through the "abstraction" approach with the semi Markov decision basis,in order to reduced state space dimension,and solve the "Curse of dimensionality" problem.Its classical algorithms were HAM,MAXQ and Option.Finally,this paper used the idea of hierarchical reinforcement learning to solve the problem of path planning and formation control of multi agent system,The main work of this thesis can be summarized as follows :(1)In this paper,a multi agent path planning algorithm was proposed which was based on the hierarchical reinforcement learning and artificial potential field,from the point of view of slow convergence and low efficiency and poor adaptability of path planning algorithm.Firstly,a target point with the maximum potential barrier potential value is zero monotonically increasing surface could be got by using the artificial potential field method after the difference to the standardization of environmental construction of prior knowledge.Finally,Multi agent made it possible that the algorithm had the task of hierarchical and good online learning ability and automatic classification of molecular tasks,which based on building a priori knowledge with the idea of hierarchical reinforcement learning.The algorithm has been verified in the experiment of the taxi problem and in the typical 3D simulation platform.The results showed that the algorithm has a fast convergence speed and stability for multi agent.(2)In view of the current stage of multi agent formation control,there were many shortcomings,such as poor environmental adaptability,agent self-learning ability and slow convergence speed.In this paper,a multi agent dynamic formation method was proposed,based on hierarchical reinforcement learning and CMAC neural network.Firstly,the whole task was divided into the root task collaboration layer through the "abstract mechanism",which help to reduce the dimension of the state space and the task decomposition of the task selection layer and the basic action layer of the three tasks in n multi Agent dynamic formation.Secondly,the CMAC neural network can be used as a state generalization method and Q-learning method in hierarchical reinforcement learning,the CMAC space storage was reduced by partitioning the state variable.Then,by using several CMAC functions,the Q functions of the degraded state are approximated by the state of philosophy to realize the generalization of the continuous state,and to speed up the learning rate of the algorithm.Finally,the algorithm was proved to be feasible in the 3D simulation platform code,and fast and stable in MATLAB.