| While e-commerce platforms and social media provide hundreds of millions of users with convenient life,work,social interaction and entertainment,they are also generate massive amounts of text data in the Internet space,among which user-generated data usually contains emotions and emotions tendency.Collecting,sorting,and analyzing the opinions and tendencies contained in these data can be used for user portraits,public opinion analysis,personalized recommendations,etc.,which has extremely high commercial value.Therefore,how to efficiently and accurately obtain effective information in massive texts and perform sentiment analysis has become an urgent need in academia and industry.Sentiment analysis,also known as opinion mining,is one of the important tasks in natural language processing.It mainly analyzes,processes,induces and infers sentimental subjective texts.At present,pre-training language models have achieved good results in sentiment analysis tasks,but there are still problems such as high computational overhead and long training time.At the same time,sentiment analysis tasks usually use a single-task learning method,which will cause the inherent noise of the data to be difficult to eliminate and over-fitting.Therefore,how to compress the model under the premise of ensuring performance,obtain higher accuracy through lightweight models,effectively reduce over-fitting,and eliminate data inherent noise is of great significance.This paper proposes new methods for the above problems.The main work is as follows:1.Aiming at the problem of long training time and high computational cost of pre-training language models,this paper proposes a sentiment analysis method SA-ALBKD based on ALBERT and knowledge distillation.This method uses the ALBERT model with strong characterization ability as the teacher model;then pre-training guidance and paired parameter sharing are performed on the student model Text-CNN to improve the complexity of the model and strengthen the learning ability of knowledge;finally,the cross-entropy loss function is used Combine soft and hard labels to distill the knowledge of the student model to get the final sentiment analysis model.A comparative experiment on the public data set of sentiment analysis task proves the effectiveness of the proposed method.2.Aiming at the problem that it is difficult to eliminate the inherent noise of data in singletask learning,this paper proposes a sentiment analysis method SA-MTLKD based on multitask learning and knowledge distillation,which models regression tasks and classification tasks together,and combines knowledge distillation The idea is used in sentiment analysis.First,BiLSTM and CNN are used as the parameter sharing layer to obtain the correlation between the texts and extract the local features of the texts,and then use the task weight distribution and knowledge distillation methods to jointly train the multi-task learning model to obtain the final sentiment analysis model.The experimental results on the public data set show that the accuracy of this method is up to 9.69% compared with traditional sentiment analysis methods. |
