Research On Pre-training Model For Text Analysis Based On Semi-supervised

In recent years,the technology of deep learning has been widely used and developed,meanwhile it is also naturally applied in the task of natural language processing,and the resulting pre-training model has also been widely used.Pre-trained models play an extremely important role in both sentence extraction and text sentiment analysis.With the development and maturity of pre-training models,the use of large-scale corpora for unsupervised pre-training models has also been proven to be the optimal way to effectively train data models.The pre-training model brings natural language processing into a new era,and pre-training technology has become an irreplaceable mainstream technology in the field of natural language processing.Recently,some transformer-based architectures,such as BERT models,etc.have provided impressive results in many natural language processing tasks.However,the benchmarks used by most models are composed of a large number of samples,sometimes even hundreds of thousands of samples,which results in a huge time cost to obtain high-quality label data in many practical scenarios.A promising approach to unsupervised learning has been proposed in image processing tasks similar to natural language processing tasks,using generative adversarial networks to process data.In order to solve the problems existing in natural language processing in this paper,after improving the above two types of models,the data processing is studied,and the semi-supervised learning method is adopted.When the proportion of labeled data is relatively small,the data processing task that can be realized only after a large amount of labeled data is completed.The data processing task that can only be realized by label data solves the cost problem of obtaining sufficient label data in the process of big data processing,at the same time,it ensures the accuracy of the experimental results,and the error loss caused is relatively small.The main work done in this paper is as follows: First,the pooling method of the model is improved,the sequence of the hidden layer is converted into a vector,the mean and maximum values are obtained along the dimension of the sequence length,and then spliced together into a vector,After the same mapping,a value is formed and activated,which effectively prevents overfitting.Secondly,adjust the number of words in the dictionary,set new hidden layer dimensions,the number of attention mechanisms,the dimension of the linear mapping of the feedforward layer and other indicators to simplify the BERT model,improve the running speed of the model and improve the model training cycle.In the meantime,the mask representation of English,numbers,and unknown characters is normalized for data processing.In the task of classifying Chinese,the error caused by special characters on the experimental results can be reduced.Thirdly,the combination of the simplified BERT model and the unsupervised model GAN model realizes semi-supervised training,which improves the accuracy of data processing.Especially in the case of a large sample size,the semi-supervised model can greatly save time and cost,but it does not.The accuracy of the experimental results is not only affected,but also even higher accuracy can be achieved.Finally,the public data sets of four different tasks are used to perform natural language processing of various tasks such as positive and negative evaluation classification.The experimental results of the model are verified by using the corresponding evaluation criteria,At the same time,the effectiveness of the model is proved.