Research On Modeling Strategy And Interpretability Of Deep Sentiment Analysis Models

Deep neural networks(DNNs)have been widely used in various fields of natural language processing(NLP),especially in the field of sentiment analysis.DNNs have the ability of feature self-learning,which is especially suitable for learning abstract,high-dimensional and complex semantic features of text.However,deep neural networks have a large number of training parameters,their decision-making process is difficult to understand like a black box.It is well-known that interpretable models are not only easy to improve but also easier to be understood and trusted by users.Therefore,many researchers have begun to study the interpretability of deep neural networks.From what has been discussed above,this thesis takes the sentiment analysis task as the entry point for exploring the modeling strategy of deep neural networks.Meanwhile,a kind of improved self-explanatory generation module is proposed and constructed to understand and explain the decision-making behavior of DNNs in human language.The main innovations of the thesis are as follows.First,a sentiment analysis modeling strategy based on multi-word embedding and multi-model fusion is proposed.Based on which this thesis compares the different effects of word embedding methods and machine learning models,and experiments with a multi-model fusion strategy to improve the sentiment analysis performance.At last,the strategy has proven the effective improvement of the sentiment analysis performance on a large-scale hotel review dataset.Compared with the optimal baseline model,the classification accuracy of the proposed model is improved by 1%.Second,a sentiment analysis modeling strategy by improving the fine-tuning language model is proposed.There are three major problems of tuning language model:too many parameters,poor transfer learning ability and limited sample input length.To solve the above problems,this thesis put four strategies to the test: freezing the language model weight,compressing language models,using convolutional networks and capsule networks to extract high-level semantic features,and modifying position embedding.Experiments on three open Chinese sentiment classification datasets show that the combination of the above strategies can obtain additional advantages in training speed and model performance compared with the strategy of fine-tuning language model.The experiment shows that the accuracy of the above method is increased by 1.5% at most,and the training time is reduced by 67%.Third,a multi-input sentiment analysis model modeling strategy based on an improved attention mechanism is proposed.The information interaction between multiple input texts is the key to multi-input fine-grained sentiment analysis.This thesis proposes a multi-input sentiment analysis modeling strategy based on the improved attention mechanism.Specifically,on the premise of dynamic word embedding,a Bidirectional Long/Short-Term Memory(Bi-LSTM)network is applied to extract context features and the related information.Meanwhile,the strategy improves the attention mechanism by setting the output of the hidden layer of Bi-LSTM as the input of attention layer and updating the attention vector according to the location information of the text interaction.Besides,it also utilizes the weight sharing mechanism to reduce the complexity of training and capsule network to make the final classification decision.Experiments on semeval2014 and Twitter data show the performance of the model with proposed strategies has an average improvement of 1% in F1 value and accuracy over other baseline methods.At last,an interpretable method is proposed for sentiment analysis based on the language model,the capsule network and the self-explanatory generation network.In detail,this thesis proposes a method of constructing parallel corpus of interpretable corpus.Then,the method uses the interpretable corpus to train a neural network which is employed to explain the results of classification and assist us in analyzing the reasons for misclassification.The results of automatic and manual evaluation on the Chinese and English interpretable SNLI corpus demonstrate that this method not only improves the accuracy by 5% compared with the baseline model,but also improves the interpretability indexes such as confusion and BLEU.In addition,as an example of the explanatory framework proposed in this thesis,it is tested on the manually annotated sentiment analysis corpus.