Research On Text Sentiment Analysis Algorithm Based On Pre-Trained Language Model

With the increasing popularity of the Internet,the text data on the Internet is growing explosively.These text data have important application values,for example,they can be used to mine users’ opinions,grasp the dynamics of public opinion,etc.Therefore,emotion analysis has become one of the key issues in natural language processing.In recent years,deep learning models have shown good performance in text sentiment analysis tasks.Compared to traditional sentiment analysis methods,deep learning models can adaptively learn features without the need for human intervention,and the model’s performance is better.Therefore,this article focuses on the study of text sentiment analysis using a pre trained language model based on deep learning.The main work is as follows.Firstly,this thesis uses the BERT model for text feature extraction.In response to the problem of the BERT model lacking the ability to capture local information,this article introduces multi-layer dilated convolution to increase the extraction of local features.Secondly,this article introduces an attention mechanism after the convolutional layer to calculate the importance of each channel,and uses the importance as a weight to weight the channel’s feature map.Based on the above improvements,the BERT-DCNNT model was proposed and experimentally verified to have higher classification performance in text sentiment analysis compared to the comparative model.Secondly,in response to the problem of large parameter count and slow training and inference speed in the BERT-DCNNT model,a pruning method is used to cut out redundant layers,and then the convolution method of hollow convolution is replaced by the deep separable convolution of distribution calculation,reducing the computational complexity of hollow convolution.Then the output features of the convolution are fused with the output features of the ALBERT model to enhance Semantic information.Finally,experiments have shown that the lightweight model reduces the parameter count of the BERT-DCNNT model,improves training and inference speed,and is more conducive to practical engineering applications.Finally,a text sentiment analysis software system is designed based on the proposed model.The user enters the text data obtained by the crawler,converts it into a vector representation through text preprocessing and representation modules,and then enters it into the trained text sentiment analysis model for classification,and finally counts the classification results.