Research On Word Embedding Neural Networks In Fine-Grained Sentiment Analysis

Sentiment analysis,as a method for mining sentiment tendencies of users in corpora,is widely used in various fields in today's society.With the development of computer technology,people tend to obtain sentiment tendencies based on attributes or functions rather than obtain rough sentiment tendencies of corpora.Therefore,aspect-based sentiment analysis is gradually becoming a popular research focus in this field.The expansion of data scale and sentiment corpora construction provide favorable conditions for word embedding neural networks to be applied to aspect-based sentiment analysis.As one of the important models of word embedding neural networks,the Bidirectional Encoder Representations from Transformers(BERT)has the advantages of strong local feature extraction capability,significant migration learning effect and high accuracy,etc.However,its application to aspect-based sentiment analysis tasks still has the following shortcomings: The statistical features of word embedding neural networks cause BERT to be sensitive to the high-frequency features in corpora,and the model results have errors due to the polarity of such features;The Transformer uses a sequence length of 512 encoding input,and variable length input cannot be used,which causes the BERT to lose some sentiment information of the long-text corpora during training and predicting process;Large-scale corpora are unstructured with redundant corpus and interfering corpus,which leads to the waste of computing overhead in BERT's training.Therefore,based on the traditional BERT,the above shortcomings are addressed respectively,in terms of improving the accuracy of models,expanding the applicability of models and reducing the training overhead,and improving them accordingly.The improved model was applied to the corpora for aspect-based sentiment analysis experiments,and the experimental results were compared with existing methods,all of which achieved high accuracy.The specific research in this paper is as follows:(1)The Information-gain Association-vector Bidirectional Encoder Representations from Transformers(IAS-BERT)is proposed to address the problem that BERT is sensitive to the polarity of high-frequency features in the corpus.Information-gain can measure the correlation between a feature and sentiment tendency.The association-vector focuses on internal features between consecutive inputs to optimize random coverage training of[mask].Adopting the feature polarity balance weight can effectively balance the high frequency and low association feature.It is possible to avoid gradient disappearance by optimizing the initial value generating method of the feature matrix.IAS-BERT was applied to the corpora SST-2,Twitter?senti,DMSC and Chn Senti Corp for aspect-based sentiment analysis experiments,and the results show that the new model outperforms other models in most cases.(2)The Sentiment Semantic Graph Bidirectional Encoder Representations from Transformers(SG-BERT)is proposed to address the problem of sentiment information loss of long-text corpora by BERT.Using Bi-directional Long-Short Term Memory(Bi LSTM)as a word feature extractor and a sentence feature extractor to generate sentiment semantic graph.The adaptability of models to long-text corpora can be improved by aspect-based sentiment analysis in the form of sentiment semantic graph.SG-BERT is applied to the long-text corpora IMDB and Dp?senti for aspect-based sentiment analysis experiments,and the results show that SG-BERT improves the adaptability to long-text.(3)The Concept Lattice Reduction Bidirectional Encoder Representations from Transformers(Concept-BERT)is proposed to address the problem of the wastage of computing overhead in training of BERT.By analyzing the structure of corpora,concept lattice association rules are used to reduce redundant corpus and interfering corpus.Set the feedforward structure and optimized loss function to reduce the deep network computational overhead.In this approach,it is possible to reduce the training overhead while maintain high accuracy.Applying Concept-BERT to the corpora SST-2,Twitter?senti,DMSC and Chn Senti Corp for aspect-based sentiment analysis experiments,and the results show that Concept-BERT can effectively reduce the training overhead.