Soft Sensor Of Industrial Data Based On Pre-training

Soft sensor is a measurement method to predict the variables to be measured by building a mathematical model between the easily measured process variable and the quality variable,which is then used to predict quality variable.As a typical chemical process,synthetic urea,like all chemical processes,faces the problem of only a small amount of labeled data.It is difficult to build good soft sensor models using only a small amount of labeled data.In contrast,industrial processes contain a large number of unlabeled,weakly labeled data.How to use weakly labeled and unlabeled data to help improve labeled soft sensor modeling is the focus of this paper.This paper studies from the perspective of pre-train model:(1)The traditional pre-train fine-tune method is an effective paradigm that can deal with small labeled data learning.When there are very few labeled samples,the algorithm trained by this paradigm faces the problem of over fitting.Meanwhile,the algorithm under the pre-train fine-tune paradigm needs complex design according to the specific task to deal with this difficulty.Therefore,this paper proposes a new training paradigm by combining pre-train with MAML.(2)For weak label data in industrial synthesis process,this paper proposes a weak supervision pre-train MAML algorithm based on multi-layer perceptron.The multilayer perceptron learns features from a large number of weak labeled data,and then transfers as a feature extractor to a small number of labeled data for MAML training.Compared with the model trained by pre-train fine-tune paradigm,the model reduces the over fitting problem.(3)In view of the fact that there are often only unlabeled data in industrial process,a generative self-supervised pre-train MAML algorithm is proposed in this paper.A generative self-supervised pre-train feature extractor encoder is used,and then supervised MAML training is used to train the attention layer and task specific layer of the decoder on the labeled data.The algorithm can deal with the inconsistency between the sampling frequency of quality variables and the sampling frequency of process variables.At the same time,the training strategy of MAML is further explored according to the structure of encoder-decoder network.(4)In the process of industrial synthesis,there are often unlabeled and weakly labeled data simultaneously.So as to make full use of all data to help train the model,this paper proposes an algorithm based on multi-task pre-train MAML,pre-train on unlabeled and weak labeled data simultaneously,and transfer the learned features to a small number of labeled data for MAML training,and prove the rationality of this method.