Reinforcement Learning Feedforward Controller

In the current industrial process control system,linear controllers still account for the vast majority.However,the actual industrial systems are mostly nonlinear,and the actual industrial problems such as switching conditions,equipment aging and so on are challenging to the traditional controllers.It is of great significance to study more intelligent and adaptive control algorithms.With the development of machine learning algorithm,the reinforcement learning algorithm based on deep learning and machine learning makes the adaptive control of nonlinear system appear a new research direction.Reinforcement learning is a control algorithm with self-decision ability.It has similar human learning ability through exploration and trial and error.It can constantly improve its own strategies through learning,so it has excellent environmental adaptability.Like factory training new workers,it is necessary to strengthen learning algorithm before it has excellent control ability,it needs a long time complex training process.Although it has adaptive ability,there is a possibility of negative impact on the control system in the process of learning transition.At the same time,when the controlled object has time-varying characteristics,the learning transition process of the pure reinforcement learning algorithm will bring robustness problems to the algorithm.In order to solve the problems of difficulty and poor transition of reinforcement learning in process control,this paper proposes an adaptive compensation control algorithm based on reinforcement learning,and discusses its application in process control of nonlinear systems.The main work of this paper is as followsFirstly,for typical nonlinear systems,the control scheme of reinforcement learning algorithm is designed,and the performance of different depth reinforcement learning algorithms in the optimization control of nonlinear systems is studied.The effectiveness of the reinforcement learning algorithm is proved,and the existing problems are brought forward.Secondly,the research uses feedforward structure to reduce the difficulty of intensive learning and training.Through the feedforward feedback structure,the reinforcement learning is used as the external optimizer to retain the feedback loop of the original process control system.Then the control problem is simplified to optimization problem.and the convergence speed of reinforcement learning is accelerated.Finally,compared with the traditional reinforcement learning method,the simulation experiment is designed to prove the superiority of the method.Meanwhile,considering the time-varying characteristics of process control,the traditional reinforcement learning algorithm will have the transition problem to adapt to new objects.In view of this problem,the existence of feedback loop in the proposed method will improve the robustness of the control system.The simulation experiment is designed to verify the effectiveness of the time-varying object method.Then,considering the complex process control system,it is difficult to establish accurate simulation model for the training of reinforcement learning algorithm.A reinforcement learning algorithm based on network supervisory control algorithm is proposed.The strategy network is constructed according to the historical data of process operation by the method of network supervisory control algorithm,and then optimized by the reinforcement learning method.The effectiveness of the method is verified by simulation experiments.Finally,the paper summarizes the content of the full text and puts forward the problems and future research prospects of the method.