Text Sentiment Analysis Based On ALBERT-BIGRUATT

Sentiment classification has always been an important part of natural language processing,and its purpose is to find out the opinions and attitudes of users on certain issues,public opinions,or products.Sentiment analysis is widely used in many aspects,such as public opinion analysis,venture capital,etc.Traditional sentiment classification includes the method of sentiment dictionary and the method based on machine learning.Among them,the accuracy of the emotional dictionary method is closely related to the quality of the emotional dictionary,while the method based on machine learning requires manual identification,construction,and extraction of data features.With the continuous development of deep learning related theories and models in recent years,many researchers have used deep learning for natural language processing related tasks,and have made great progress in many tasks.In 2018,the BERT(Bidirectional Encoder Representations from Transformers)pre-model proposed by Google has excellent results in many Natural language processing related tasks,completely changing the research model of natural language processing.Many researchers have devoted themselves to this new pre-model research.What’s more commendable is that BERT has good performance in different natural language processing tasks,which shows that it has a wide range of versatility.Many tasks including sentiment analysis can directly improve the effect through this model.However,the shortcomings of this model are also very obvious.Although the effect is outstanding,a large number of parameters and a huge model hinder its further development.The work done in this article mainly includes the following three aspects:(1)Construction of data set for Chinese sentiment analysis.In response to the current problem of the small number of data sets for Chinese sentiment analysis and uneven quality,this article combines the fact that the demand for online education software has increased during the epidemic,and selects a software review of a certain education software for data preprocessing,and establishes a new data set for Chinese sentiment analysis.(2)Aiming at the problem of too many BERT model parameters and too large model,in 2019 Google proposed an improved model based on BERT-ALBERT(A Lite BERT).The model proposes two methods to reduce memory,while at the same time improving the training speed.Based on the ALBERT pre-training model,combining the bidirectional gated recurrent neural network and the attention mechanism,this paper proposes a new sentiment analysis model ALBERT-BIGRUATT,which reduces the model parameters on the basis of ensuring accuracy as much as possible,so that The model parameters are only about one-ninth of the BERT model parameters.Semantic features are extracted through the bidirectional gated recurrent neural network(BIGRU),and then the key information is extracted by introducing the attention mechanism.Experiments show that compared with general sentiment analysis models,this model has better effects and better performance.(3)Finally,based on the above sentiment analysis model,an sentiment analysis system including data preprocessing,model training,sentiment prediction and other functions is designed and implemented.The system allows users to choose different models for training according to their own data,hardware conditions,and acceptable time cost,and through the trained models,perform emotional predictions on one or more texts.