Researches On The Curse Of Dimensionality In Reinforcement Learning

The "curse of dimensionality" problem,which means the states sPace increases exponentially with the growth of characteristics' number,is a popular and important problem in reinforcement learning.Two methods were proposed to solve the "curse of dimensionality" problem and optimize the reinforcement learning algorithm form two different points of view.At the same time,on the basis of theories mentioned above and the SWT technique in Eclipse,a Tetris game was developed as an experiment to validate the corresponding methods.Then the reinforcement learning theory was-applied to the routing algorithm in Ad Hoc network to solve problems such as routing efficiency,energy spending and security.The main research results are concluded as follows:ⅰ.A Gaussian Processes reinforcement learning method in large discrete state spaces was proposed on the analysis of the "curse of dimensionality" problem in reinforcement leaming.The experiment result shows that the performance of reinforcement learning method combined with Gaussian Processes,such as convergence speed and final results had been improved rapidly.ⅱ.In order to solve both of the "curse of dimensionality" and slow convergence speed problem,a reward optimization method based on action sub-rewards in hierarchical reinforcement learning was proposed.Reinforcement learning method with the sub-rewards and hierarchical idea could optimize reward function and expedite the convergence speed.The experiment result shows that combined with sub-rewards and hierarchical idea,the performances of reinforcement learning were improved.ⅲ.Apply both of the methods to the Tetris game as an experiment,and the results have been analyzed to judge the efficiency of these algorithms.All the performances with different parameters were compared to advance the possible improvement in experiment results.ⅳ.On the basis of Ad Hoc network,the corresponding reinforcement learning method was applied to routing algorithm,which solved many problems integrately in Ad Hoc network such as routing efficiency,energy spending, security and adaptation.