Study On Poker Opponent Modeling Based On Bayesian Net- Work And Hidden Markov Models

The game problems have attracted artificial intelligence researchers’ interest for a long time. Many techniques in artificial intelligence have been applied successfully to reasoning and strategy games. Some of these successes have promoted the develop-ment of artificial intelligence research, and others make it possible to apply artificial intelligence techniques to similar real-life scenarios. Poker game, which is popular a-mong the people and played with certain intelligence and strategy, plays an important role in the field of game artificial intelligence.Poker game is essentially a process of reasoning under uncertainty. It provides a natural platform for testing new techniques and methods in artificial intelligence. Currently, researchers have tested a lot of methods such as Monte-Carlo simulation, game tree search, neural networks, evolutionary algorithms, Bayesian network, etc. in poker problem. While the most important part is to model the opponent effectively. The unpredictable information within poker game makes it hard for researchers to use search method to solve the poker problem. Therefore, opponent modeling has become an important tool for solving this problem effectively. Researchers have made some progress in poker opponent modeling, however, the level is still far away from the the human player’s. Current methods can not effectively capture the opponent’s patterns and styles. As a result, the poker agent can not make great progress.In this thesis, according to the deficiencies of the existing research, we propose two novel opponent modeling methods for solving the poker problem. The details are as follows:(1) We propose a method to modeling the opponent’s style based on Bayesian network in Hold’em poker. Firstly, we refine the structure of Bayesian network and use the Bayesian poker agent to infer the probability of winning of a hand. Secondly, we design the style functions to learn the opponent’s styles based on corresponding states, then the styles will be saved in the style table that we create. Thirdly, we make rules to help the poker agent make decisions with the style modeled. The results of the experiments show that the opponent’s style modeling method that we propose is reasonable and can help the poker agent win more money.(2) We propose a method to apply hidden Markov models to the opponent model-ing in Hold’em poker. Firstly, we make time series model for the opponent’s behavior tendency with hidden Markov models as the time period is the count of rounds in each hand. Secondly, we predict the opponent’s hand types with the time series model. Thirdly, we make rules to help the poker agent make decisions with predict result-s of the opponent’s hand types. The results of the experiments show that the hidden Markov models of poker can help the poker agent better grasp the opponent’s timing behaviors tendency. Agents using this method will have a good performance when playing against other elite poker agents.