Research On Feature Selection In Reinforcement Learning

The problem of dimensional disasters in large-scale state and action spaces is currently a difficult problem in reinforcement learning research.The widely used technique is the method of value function estimation.However,there is a gap between the estimated value function and the true value function.In addition to the fitting ability of the learning method itself,the source of the error also has a great influence on the error due to the quality of the selected feature of the value function.Therefore,this paper focuses on the feature selection problem of reinforcement learning,and designs some algorithms for feature selection in value function estimation.The main work is as follows:(1)Aiming at the problem of the error between the value function estimation and the true value function,this paper proposes a greedy-based wrapper feature selection method to construct a good feature for the value function.And use the method of piecewise value function to deal with the unstable characteristics of feature selection.Experimental results show that this method improves the stability of feature selection and also enhances the accuracy of value function estimation.(2)Aiming at the problem of slow feature selection in reinforcement learning,and the greedy strategy that only considers the current optimal and short-sighted issues,this paper proposes a distributed top-k greedy method to select features based on the nature of the weak submodular function.Experimental results show that this method improves the speed of feature selection and explores better features.(3)The wrapper feature selection is based on the effect of the subsequent algorithm as the criterion for feature selection,and the long training of reinforcement learning leads to the difficulty of solving the time-consuming problem even if the distributed method is used.This paper proposes a filtering feature selection method based on experience replay,which generates a reinforcement learning data set through experience replay,and then directly completes the feature selection work on the data set,and finally only needs one reinforcement learning training to test the effect of feature selection.The experimental results show that the feature selection speed of the filtering method based on experience replay is significantly faster than that of the wrapper method.