Industrial Process Modeling And Fault Classification Based On Reinforcement Learning

With the upgrading of industrial process systems and the development of monitoring tech-nology,the scale of data collected on industrial fields is increasing and the data types are di-versified,more and more complex data characteristics and process characteristics are emerg-ing.Meanwhile,the innovation of science and the big data processing,analysis and model-ing capabilities based on artificial intelligence technology provide new ideas and directions for data-driven industrial process modeling research.Focusing on the actual task requirements of industrial process modeling and fault classification,this paper focuses on key and difficult prob-lems such as semi-supervised and imbalanced data characteristics,nonlinearity and dynamics in process characteristics,and proposes an industrial process modeling method based on re-inforcement learning.Finally,the validation experiments of methods are conducted in actual process data.The main research contents of this paper are as follows:(1)A general trainable pseudo-label generator based on deep reinforcement learning is proposed for the label generation problem of semi-supervised data in industrial processes.The correctness of the generated pseudo-labels is judged by the classification performance of the validation dataset,the policy gradient loss function is used to optimize the pseudo-label genera-tor,and a sequential decision-making process of interactive iteration between the pseudo-label generator and the validation dataset is constructed.Deep reinforcement learning solves the non-differentiable problem of generating discrete labels,improves the quality of pseudo-labels and corrects confirmation bias.This generator forms a new model architecture,which is different from independent non-iterative models and improves the performance of ordinary models in semi-supervised data.(2)Aiming at the cost-sensitive problem of imbalanced data in industrial process,a dy-namic cost-sensitive classifier based on Actor-Critic model is proposed.By designing an ap-propriate reward function,using an action network with policy gradient loss,and introducing a new cost matrix into the evaluation network,a framework for learning the cost of different sample classifications with a new Actor-Critic model is constructed.The process of learning sample weights by the Actor-Critic model is related to the actual classification performance of the samples,so that Actor-Critic can learn the optimal cost matrix,which solves the problem of imbalanced fault classification.The cost-sensitive method relies on expert experience,and it is difficult to set the cost matrix.For the problems of high-dimensional variable data and nonlinear processes,the neural network is used as the network model of reinforcement learn-ing to perform dimensionality reduction and nonlinear feature extraction.The proposed novel cost-sensitive learning strategy can adaptively learn the cost matrix and dynamically generate sample weights in the process of model learning,so as to effectively improve the performance of imbalanced fault classification.(3)Aiming at the cost-sensitive,large number of classes,and the difference between het-erogeneous and homogeneous classes of extremely imbalanced data in industrial processes,an reinforced knowledge distillation algorithm is proposed.Through the teacher network,the soft target of the learned knowledge is transferred to the student network.The fine-grained classi-fication framework is used to refine complex tasks into multiple sub-tasks.The fine-grained classification framework based on hierarchical clustering and knowledge distillation model is constructed.This method solves the problem of the large number of classes,the difference between heterogeneous and homogeneous classes in the extremely imbalanced problem,and obtains better classification performance.The sample weights are learned by reinforcement learning.The framework for improving fine-grained classification models based on knowledge distillation strategy and policy gradient reinforcement learning is constructed,which not only solves the difference between heterogeneous classes in the multi-class imbalanced classification problem,the problems of different sample importance and sample dispersion in the homoge-neous class,the indistinguishable samples in the homogeneous class,but also the intra-class distance becomes smaller and the inter-class distance becomes larger.For the imbalanced fault classification task,the method has outstanding classification performance.(4)A new general imbalanced sample selection framework based on deep reinforcement learning is proposed for the sample selection problem of imbalanced industrial process data.The problem of removing outliers and selecting valid samples is transformed by MDP.It takes the selection of a subset of samples as the action,the feature information and label information of the training set as the state,the classification performance on the validation dataset as the reward,and the maximization of the reward of sample selection as the goal.For the multi-armed bandit problem,a single-state markov decision process for sample selection in the training set is constructed,which solves the problems of unstable performance,difficulty in designing sample weights,and low method universality in the sample sampling problem.Finally,REINFORCE loss is used to optimize the sample selection process.As a data-level method,this method has certain validity,stability and transferability.(5)Aiming at the sample selection problem of extremely imbalanced data containing ran-dom noise in industrial processes,an integrated imbalanced sample selector based on the Soft Actor-Critic model is proposed.By introducing the idea of integration,the sequence process of boost is skillfully combined with the trajectory in reinforcement learning,the learning pro-cess of the sampler is modeled as a sequence decision process,the error density is proposed for difficult sample mining,and the model is carried out by interactive iteration.This training process reduces the model error and bias,improves the stability of the model,obtains better classification performance,and improves the generalization ability of the model.Meanwhile,this method improves the way of actions,which not only reduces the amount of calculation of model parameters,but also makes the method have certain effectiveness and practicability.