Research On Sentiment Analysis Of Chinese Text Based On Transformer

In recent years,with the widespread popularity of mobile devices such as mobile phones and the vigorous development of social media software,a large amount of subjective linguistic information which includes the emotional tendency of publishers has been generated on the Internet.Understanding and analyzing the emotional tendencies contained in these information is of great significance to areas such as online public opinion monitoring and business investment choices.With the birth of BERT in 2017,the research status of natural language processing has been completely changed,and almost all fields of natural language processing,including sentiment analysis,have entered a new stage.Although the effect of BERT is great,the huge model size and parameters severely hinder the online application of such models,and it is imperative to reduce the model size while retaining the model effect as much as possible.This thesis is about the current development of sentiment analysis and deep learning,and has carried out the following work:1.Aiming at handling the problem that the open source Chinese sentiment analysis dataset is scarce and of poor quality,based on the open source data set,the senti ment tendency of the text is recalibrated and subtracted.The method established a suitable Chinese Weibo sentiment analysis dataset for further use.2.Applying ALBERT to sentiment analysis,the sentiment analysis technology based on ALBERT is proposed,a new sentiment analysis model ALBERT-FN-M was established,and the comparison with the traditional word vector sentiment analysis model is done.The experimental results show that the model based on ALBERT has a significantly improvement compared with the traditional word vector model.The size and the amount of parameters are also greatly reduced compared with BERT model.3.By applying the method of knowledge distillation,the effective information of the large model ALBERT-FN-M was successfully distilled into small models such as BiLSTM.the microblog text sentiment analysis model ALBERT-D-Bi was proposed.The method of knowledge distillation on a large sentiment analysis model is proposed.Compared with the traditional word vector model,the performance is significantly improved.Compared with the original large model,the training time and inference time are reduced significantly.