Data-driven Operation Optimization Control Of Industrial Process Based On Reinforcement Learning

It is difficult to accurately model productive processes and describe relationship between operational indices and controlled variables for modern industrial processes.However,traditional industrial processes use artificial experience to adjust the set value of the controller in the form of open loop,and the manual operation does not have high quasi-determination and follow-up,so the economic benefits are poor and may even lead to the occurrence of faulty conditions.Under these circumstances,how to design the setpoints by using only data generated by operational processes for optimizing operational indices,without requiring the knowledge of model parameters of operational processes,poses a challenge on operational optimization control.This paper focuses on a class of industrial processes that can be linearized near the steady states and take different time scales adopted in the operational control loop and process control loop into account.In this context,a Q-learning based suboptimal setpoints learning algorithm is proposed to learn suboptimal setpoints by utilizing only data,such that the operational indices can track the desired values in an suboptimal manner.A simulation experiment in flotation process is implemented to show the effectiveness of the proposed method.The main content and the contribution in this paper are as followed.Firstly,describe the double-layer operation feedback control problem,model and design the operation optimization controller for the different time scales of the operation control loop and the bottom control loop,so that the control layer outputs the tracking setpoints.Secondly,to our best knowledge,data-driven industrial process operational optimization control has not been fully investigated.This paper proposes a Q-learning based suboptimal setpoints learning algorithm and a state-observer based Q-learning algorithm to learn the optimal setpoints by only using data for industrial processes to drive the operational index to the desired value by forcing the control outputs to the optimal setpoints.Thirdly,we use the flotation process as a simulation example to make some simulation experiments in the Matlab environment and compared with the traditional operation control method.The conclusion proves the validity of the two algorithms proposed in this paper.