Applying Temporal Difference Learning In Chinese-chess Computer Game

Compared with International-chess, Chinese-chess is a more complex board game. Chinese-chess is played on a square grid containing 10×9 intersections with 32 pieces.In Chinese-chess computer game (CCCG), the most time-consuming aspect of building a high performance game playing program is the design, implementation and tuning of the position evaluation function.In this thesis, a three-layer fully-connected feed forward neural network is designed as a position evaluation function. Temporal difference learning (TDL) is a reinforcement learning algorithm, which uses the difference of a pair of successive position-values to incrementally update the weights. Based on the three-layer neural network with single output, we derive a new weight updating rule for applying TD(λ) in CCCG. Starting with random initial weights between-0.5 and 0.5,the neural network is trained through the new rule on the professional game records.In the training process, each professional game record is learned iteratively by the neural network until the evaluation value of the position in professional game record becomes stable ultimately. In the experiments, we validate that our learned evaluation function is feasible and effective.Furthermore, we obtain three main relationships between(1)the number of nodes in hidden layer of neural network, (2) the parameterλ,(3) the learning rate a,and the performance of the evaluation function in the process of training different neural networks through temporal difference learning.