Research On Deep Reinforcement Learning Methods For Text Games

With the continuous development of deep reinforcement learning technology in the field of video games,more and more video games of varying difficulty are mastered by deep reinforcement learning agents,which can even be comparable to human professional gamers in some video games.Compared with video games,text games based on text language are due to their special game methods and low market popularity.As a result,there is less research in the field of text games than in the field of video games.Based on the above background,this article will focus on the less popular text games.In the previous related research on text games,most of the methods of modifying the deep reinforcement learning agent model were used to try to improve the game performance of the agent in the text game environment.However,most of these agent models have limited representation capabilities.Moreover,most of these studies ignore some of the characteristics of the text game environment itself,which limits the field of vision of the agent model in the process of exploration and learning.This paper focuses on the existing research work on the limited representation ability of the agent model and the limited field of view of the agent.It mainly includes the following three aspects:First of all,in response to the problem of limited agent representation capabilities in previous studies,this paper proposes the DSRRLM method.This method improves the representation ability of the agent model by introducing deep successor representation technology in the process of agent model design,and uses an interactive function to calculate the Q value in the prediction stage of the model.At the same time,this article also uses the word embedding model GloVe to pre-train the word vector.The GloVe model can make the text retain as much semantic information as possible in the vectorization process.In the experience replay stage of the deep reinforcement learning method,this paper adopts the priority experience replay algorithm to speed up the convergence speed of the model.In order to verify the feasibility of the reinforcement learning method based on deep subsequent representation technology in text game task,this paper compares this method with previous methods through experiments.The experimental results show that,under the same experimental settings and comparison standards,the text game agent model based on deep successor representation designed in this paper is better than other comparison models.Secondly,considering that most previous studies have ignored the game characteristics of text games themselves,this paper proposes the GNNRLM method.This method further integrates contextual information on the basis of the DSRRLM method.This method of integrating contextual information can effectively expand the agent's field of vision,allowing the agent to have a clearer and more accurate judgment on the current game state.In terms of feature extraction of contextual information,this paper constructs contextual information as graph structured data,and uses graph convolutional neural network and graph attention network respectively to extract features of contextual information.In order to verify the effectiveness of the model designed in this paper and explore the gain effect of context information for text game agent,this paper takes the original model that does not use contextual information as the baseline model,and compares it with the GCNAM model and GATAM model designed in this paper.The experimental results show that the two models designed in this paper can be effectively applied to the text game environment,but the level of game technology that can be achieved by the two is different.It also proves that the contextual information used in this article can indeed benefit the text game agent.Finally,this article still focuses on the contextual information to carry out further research.In order to give full play to the gain effect of context information,this paper proposes the TSNRLM method.This method constructs context information as time series data,and uses a variety of time series network structures with modeling capabilities on time series data to extract the characteristics of context information.In this paper,four types of agent models are designed based on CNN(one-dimensional),RNN(GRU,LSTM),and TCN structures.This paper fully explores the gain effect of contextual information constructed as time series data through experiments,and at the same time fully explores which type of time series network is more suitable to be applied to the model design of text game agents.Experimental results show that it is more effective to construct contextual information as time series data than to construct graph structure data.At the same time,it also shows that among the four agent models based on time series network designed in this paper,the one-dimensional convolution-based agent model has the most superior game performance.