Research On Imperfect Information Machine Game Based On Deep Reinforcement Learning In 3D Game

Since the concept of artificial intelligence has been proposed,the machine game plays an increasingly import role in a lot of field.Among them,it is regarded as one of the most challenging research directions and the litmus test of artificial intelligence theory.Machine game can be divided into perfect information machine game and imperfect information machine game.The characteristic of imperfect information machine game is that agent can't obtain all the information in the game process.Many real-world decision-making problems can be abstracted as imperfect information game problems,such as military game,commercial competition,network security,financial control and other problems.Therefore,it is great practical significance to study the imperfect information machine game.The traditional method of solving the imperfect information machine game problem is partially observed Markov decision process model and reinforcement learning algorithm.However,the reinforcement learning algorithm can't guarantee convergence in imperfect state and high latitude state space.Only through limited data and repeated testing can't traverse all the state.In this paper,we combine deep learning with reinforcement learning and the state-action value function in reinforcement learning is replaced by a deep learning network.We use deep reinforcement learning algorithm to solve the related problems in the domain of imperfect information machine game.Traditional methods of reinforcement learning need to extract features manually.It is difficult to find the internal relations between features.Besides,training requires a lot of domain knowledge,which makes poorly scalability.Deep reinforcement learning can use the original game screen as input and complete the end-to-end training.It realizes the process of agent self-learning.Aiming at the problem that long-time historical information can't be considered in the decision-making process of deep reinforcement learning algorithm,we propose to add the long-short term memory model to the deep reinforcement learning algorithm.At the same time,there is a problem that Q value of the suboptimal action is usually over-estimation in traditional deep reinforcement learning algorithm.This paper refer to double Q learning,which decouples the Q value estimation in the original deep reinforcement learning and the DDQRN network is proposed which is conbined by the DQRN network and the double Q network.Due to the inefficient training and slow convergence of the improved deep reinforcement learning network,this paper` propose the priority search in experience replay structure.In order to improve the battle efficiency of the agent,the network structure is divided in to three parts which is composed of visual neural network,map navigation strategy reinforcement network,battle reinforcement network.This paper achieve a high intellectual level agent in Vizdoom game,which is similar to the real life on firstperson perspective.Compared with the traditional reinforcement learning algorithm,the agent achieves a higher level of game.