A Study Of Hierarchical Reinforcement Learning Algorithm Based On Fuzzy Clustering

Reinforcement learning is an important method of machine learning, because it does not need the environment model and improves its behavior policy with knowledge obtained by trial-and-error interaction with the environment. Therefore, reinforcement learning has the ability to self-learning and online learning. However, in the large state space of application problems, the "dimension curse" problem hasn't been solved yet. Hierarchical reinforcement learning introduces an abstraction mechanism to make state space lower the dimension, i.e., the task of reinforcement learning can be analyzed into the different layers between the abstract inside and within the abstract, and be realized respectively. Therefore, the task of learning in every layer need be carried out only in the space of low dimension. It is an effective method that solves the problem of "dimension curse", and opens a vast range of prospects in application.Hierarchical structure of hierarchical reinforcement learning can be determined in advance by the designer who depends on the knowledge of the experts, or be generated automatically. Since the complicated environment, or learning expert knowledge, it is difficult to determine any hierarchical structure in advance. For that reason, an effective method of automatic stratification has become the research focus recent years all the time. This paper makes a research and explores in the following aspects:First of all, this paper introduces the research and development on the methods of hierarchical reinforcement learning, and then deeply discusses the related theories of reinforcement learning and hierarchical reinforcement learning, and furthermore compares the advantages and disadvantages of these two methods.Secondly, this paper explores the method of fuzzy clustering, in view of the features of reinforcement learning task, puts forward a kind of improved method based on genetic algorithm, fuzzy clustering method.Finally, this paper proposes a new method of hierarchical reinforcement learning. It fuses the improved fuzzy clustering method into hierarchical reinforcement learning, carries out automatic clustering in large state space so as to make state space lower the dimension, and realized the automatic hierarchy of learning task, on the basis of the generating clustering sub-space. The results of the experiment illustrate the validity of this algorithm.