Paper-making Process Controller Design Of Reinforcement Learning

The pulp and paper-making affects every aspect of our daily life and has a great significance to parcels,documents,life and other fields in society.In the process of the pulp and papermaking,there are many complex physical and chemical varieties,which make the system has significant nonlinear,coupling and time delay,and increase the difficulty of the system control.These objects exist in the core process of paper making,and their control performance directly decides the quality of paper.As an excellent time series decision system,reinforcement learning algorithms are very suitable for the control environment of these systems.Aiming at the basic weight,moisture and ash control system of paper,this paper deeply discusses the reinforcement learning control method and the detail research of coupling and time delay of paper making process.The research and contributions of this paper are listed as following:1.Aiming at the influence of the coupling between these objects on the designed controller,using the ideal point method to transform a multi target control to single objective optimization.The control process is then can be transformed into a Markov Decision Problem(MDP).Then the Proximal Policy Optimization(PPO)algorithm is used to solve the sequential optimal decision path of the complex internal relationships.The simulation results demonstrate that,in the control of the basic weight and moisture,the proposed method can achieve or even exceed the effect of PID control with feedforward decoupling,and proves the feasibility of using RL algorithm.2.Aiming at the impact of the trade-off between exploration and exploitation in RL on the robustness of the learned policy,combining trust region and maximum entropy reinforcement learning,by assuming a policy distribution obey the Gaussian distribution,re-derive the expression of policy performance function and optimize the objective function,and simplify the solution by clip function,which makes the algorithm easy to implement and training more faster.The experimental results show that the proposed algorithm is more robustness and generalization in the control of basic weight,moisture,and ash control than the PPO algorithm.3.Aiming at the weight allocation problem in the ideal point method,this paper proposed a solution which utilizes the multi-agent reinforcement learning(MARL)to solve the multi-objective problem,and the cooperative tasks and mix tasks under the multi-agent MDP framework are extended.The multi-agent deep deterministic policy optimization algorithm is used to solve the problem.The simulation results show that the policy learned by MARL has smaller steady state error,faster adjustment time,and can well solve the training fluctuations caused by time delay.