Text Sentiment Analysis Based On Pre-training Language Model

With the development of mobile Internet,choose the online shopping population increases year by year,also have a lot of comment text.analysis the user overall sentiment tendencies,and in view of the different aspects of the sentiment,can help other customers make a buying decision,therefore have important research value.This paper mainly carries out sentence-level sentiment analysis and aspectlevel sentiment analysis on online shopping product reviews.In the task of sentencelevel sentiment analysis,GRU sentiment classification based on pre-training language model is proposed.First,the pre-training language model of GRU unit of RNN neural network was adopted to obtain the trained language model.Then,two transfer learning methods,Frozen and Fine-tuning.Secondly,in order to solve the problem of some deviation in the traditional RNN sentence representation,the pooling layer was used to further extract text features,average and maximum pooling were done for the output vector of the hidden layer.Finally,the mobile comment text of jingdong mall is acquired through the web crawler technology and used as the experimental data set.Experimental results show that,compared with the traditional machine learning and non-pre-training model,the pre-training language model based on Frozen proposed in this paper has the highest accuracy.For aspect-level emotion analysis,the method combining the Attention mechanism is adopted.The main idea is to assign different weights to the characteristics of the hidden layer according to the given aspect information,and finally get different text representation,and then conduct sentiment classification.At the same time,aspect-level emotion analysis also adopts the pre-training modeling strategy,that is,the shallow embedded layer and the GRU hidden layer adopt the network parameters of the pre-training language model.