The Research And Implementation Of Large Space Reinforcement Learning Based On Model Knowledge

Reinforcement learning is an unsupervised learning technology, by which the agent can find optimal behavior sequence and perform from uncertain reward of envirenment in on-line learning. So reinforcement learning is recognized as an ideal technology to construct intelligent agent.Q learning algorithm is one of the most popular reinforcement learning algorighms, but the algorithm exist some problems.Firstly, Q learning algorithm not make full use of the experience knowledge in each process of reinforcement learing, convergence rate is very slow; Secondly, Q learning algorithm can't be used in the learning problem of continuous state space and continuous action space directly,although fuzzy reinforcement learning algorithm can resolve this problem in some extent, But the selections of membership function in fuzzy phase are always based on subjective experience and some necessary mathematic processes. So, error is unavoidable. In this essay, we improve and extend Q learning algorithm by combining with other learning method. The improved algorithm can be used in agent's decision-making in complex environment. The main works are as follows:(1) Based on Q learning algorithm, environment model learning is imported into reinforcement learning process. Envirenment model is learnt firstly, and then use the new model just learnt to reguide the reinforcement learning process. In this algorithm, search space is reduced, in another words, time and convergence rate are improved. We applied this method to shoot problem of RoboCup successfully. The policy of shoot problem are optimized.(2) Genetic algorithm is used for selecting membership function of fuzzy Q learning which make the fuzzy divide process more accruate and effective.Experiment is associated with kick problem of RoboCup, comparison between basic fuzzy Q learning and developed fuzzy Q learning show the superiority of this algorithm.