Hyperparameter Optimization Of Identification Algorithm Based On Reinforcement Learning

In the system identification theory,there are many identification algorithms with parameters that need to be set manually,which can be called hyperparameters,such as the decaying weight of fading memory least square method and the hyperparameters of neural networks,etc.The configuration of hyperparameters in the identification algorithm significantly affects the identification accuracy.And the identification of different systems often require different hyperparameter configurations.Therefore,setting the hyperparameters reasonably is the key to achieving high-precision system identification.In this paper,a series of optimization methods for the hyperparameters of identification algorithms based on reinforcement learning was proposed.Hyperparameters optimization in identification algorithms is a typical black box optimization problem.So model-free reinforcement learning algorithms can be used to solve this problem.Since the decaying weight of fading memory least square need to be selected in a continuous space,DDPG algorithm was chosen to solve the decaying weight optimization problem.In order to improve the search efficiency,this method is modified as follows:First,process the instant reward to limit its upper and lower limits.Then,select the value of the current decaying weight with the best performance as the initial state at the beginning of each episode.These improvements can ensure that the reinforcement learning agent learns effectively in each step and searches more in the high-performance values.In order to verify the performance of the forgetting factor optimization method based on DDPG,conduct several optimization experiments on the forgetting factor in the fading memory least square algorithm.The experimental results shows that this method can effectively optimize the decaying weight for different identification objects,making sure the fading memory least square method to realize high-precision system identification.In order to optimize the hyperparameter configuration of a neural network,select the hyperparameter to be optimized and its candidate values at first.Then,use the Q-learning algorithm to solve this hyperparameter optimization problem.In order to speed up the progress,do not set fixed initial state and terminal state for each episode in Q-learning algorithm.To verify the performance of the neural network hyperparameter optimization method based on Q-learning,this method is used to conduct several optimization experiments on the hyperparameters of a BP neural network.The experimental results show this hyperparameter optimization method can effectively optimize the network hyperparameters for the nonlinear systems identification,so that it can achieve highprecision system identification.