On-line Human Simulated Intelligent Control Tuning Using Reinforcement Learning

Human Simulated Intelligent Control (HSIC) is based on recognition of multiple controllers and multi-mode control structure. One notable characteristic of control is the change between the proportion (closed loop) and keeping (open loop) mode. It has broken the control information processing of the traditional control theory with single map structure and properly solved the contradiction among the stability, accuracy and fast. It can solve the control feasibility of complex system.However, because of its multi-mode controller and the multi-control structure, it has many feature and control parameters. So, it's hard to design the controller. In addition, the system can change itself by environment and the existence of uncertainty. So the control parameters can't keep the same value on the whole processing. It must be regulated on time. The AVR system using HSIC learning parameters can improve high-quality of speed and real-time control.Online learning and optimization parameters play an important role in the control. The biggest difference of learning parameters between online and offline is that online learning parameters learning can adapt to dynamic environment, but offline learning parameters are only adapt to static environment. The methods of online learning parameters include: Simulated Annealing (SA), Particle Swarm Optimization (PSO), the Simplex and Reinforcement Learning method. This paper bases study of AVR system and use reinforcement learning online learning the parameters of HSIC.Continuous action reinforcement learning automata (CARLA) is a kind of reinforcement learning,which gets the parameter value in the continuous space. The algorithm uses a continuous probability density function (CPDF) to deal with every decision variable. It is through several iterations to modify the parameters and will eventually converge to a stable value of parameter. Each modified process is determined by a value of the reinforcement signal.This paper achieved to use CARLA online learning the parameters of HSIC with multi-mode control structure and the hierarchical structure. At the end of this paper, chose one system to test the algorithm and has accomplish both online optimizing HSIC parameters based on CARLA and PID control parameters based on CARLA. In addition, this system also uses genetic algorithm optimizing parameters. Under the different controllers, HSIC is better than PID controller for the system. Under the same controller, optimizing parameters with CARLA is much better than using GA.