The Mutli NPC Collaborative Research Based On Reinforcement Learning Of MMOG

Artificial Intelligence technology which are currently applied in MMOG(Massively Multiplayer Online Games)is simple. Reinforcement Learning algorithm can make game intelligence more complex. Because of the real-time characteristic of MMOG, Reinforcement Learning algorithm can not be applied well. So it is very important to research on Reinforcement Learning method appropriate to MMOG.This paper researches respectively on the action type and strategy-based Artificial Intelligence technology in MMOG, based on the study of the Reinforcement Learning related technology. This paper completed the following two works:1. For the problem of action type Artificial Intelligence in MMOG, to propose the Heuristically Accelerated Evaluated Q-Learning (HAE-QL). The method uses a heuristic function to influence the choice of actions during the learning process, and an evaluation function to evaluate the selected action, which can reduce the unnecessary exploration and improve learning efficiency. To ensure the effect of this method, the heuristic function and evaluation function are calculated by the Q function. Meanwhile, to propose a Kalman Filter dead reckoning algorithm based on HAE-QL. It can not only ensure the NPC correct learning under delayed communication, but also ensure the effective learning.2. For the problem of strategy type Artificial Intelligence in MMOG, to propose MMRL (MMOG Reinforcement Learning) algorithm. In the algorithm, the choice strategy of the NPC is based on the trust of the choice strategy of other NPC. It is unlike most existing algorithms which are based on the trust of the actions. The algorithm only records the situation of the NPC learning failing, because the fail situation is much less than the success situation, the space of the history knowledge will significantly reduced. It improves the efficiency of the algorithm. Meanwhile, this algorithm uses the history records to assess the NPC learning action, which more advance the efficiency of the algorithm. The experimental results show the HAE-QL method and MMRL algorithm can significantly improve the NPC's confrontation and provide good support for the NPC learning in MMOG.