Computer Poker Based On Monte-Carlo Tree Search

Computer gaming is an important platform to test the level which artificial intelligence can achieve. The research on computer game helps construct expert system and machine auxiliary decision system. Early research of computer game focused on games with deterministic complete information. Recently, games with non-deterministic imperfect information attract academics’ attention gradually, because it is more close to the real world.Texas Hold’em is a non-deterministic imperfect information game. Because of the simple rules and the variations in playing, Texas Hold’em becomes a typical case of the kinds of games and another hot spot in computer game research. In machine learning, Monte-Carlo tree search is a game-tree-search algorithm which is integrated with Monte-Carlo method as its evaluation function. Without introducing too much domain knowledge in estimating game status, this method has great scalability. Multi-armed bandit problem is a machine learning model abstracted from the problems of multiple decision selection. The UCB strategy for solving this problem can concert exploration and exploitation. Monte-Carlo tree search algorithm based UCT which is integrated UCB strategy has been shown to enhance the level of the computer game engine greatly.In this paper, we describe the related features of Texas Hold’em to design a Texas Monte-Carlo tree. We also design the simulation, selection, and update strategy for the nodes in the game tree. In the paper, we implement a Texas Hold’em engine with C++programming language, object-oriented technology and the design pattern ideas.In this paper, the domain knowledge related to Texas Hold’em is introduced for the engine based on Monte-Carlo tree search algorithm. The domain knowledge includes Bucketing hand abstraction and the opponent model which could simulate the opponent and estimate the probability distribution of the opponent’s cards. Finally, we propose a conception of conservative. Using this conception, we can use a more loose strategy to transform the betting behavior of engine.We design four experiments. First we test the performance of the pure Monte-Carlo tree search engine. Then we analyze the output log and find that the tight betting strategy has a negative impact on performance. We also test the influence on performance after we use a loose betting strategy. At last we verify the effect on performance, when we integrate the opponent model into the engine.In the end, through the analysis of the experimental results, we can confirm that Monte-Carlo tree search algorithm can be applied to solve the games with non-deterministic imperfect information. At the same time, in Texas Hold’em, the integration of domain knowledge and opponent model can improve the performance of the Monte-Carlo tree search engine.