Interpretable Research And Application For Deep Text Classification Model

In recent years,the task performance of Natural Language Processing including text classification has been greatly improved with the rapid development of deep learning.However,due to deep neural networks belong to a black box architecture,the process of model's decision-making is not transparent,so the model is poorly interpretable.At present,more and more researchers are focus on interpretable machine learning.As linguistic intelligence is the core module for machine understanding of humans,the interpretability research of NLP is gradually becoming one of the future development directions.The existing interpretability methods are mainly divided into two categories.One is the post-hoc interpretability which using the interpretation method to explain the model after the model training,and the other is the ante-hoc interpretability which using the built-in interpretability model.This thesis proposes two interpretable methods for deep text classification model.The first method is based on adversarial examples which is post-hoc interpretable.This method first uses contextual word vectors generated by self-encoding language models.Using the interpretability of the gradient in the word embeddings,we look for interpretable perturbations in the model backpropagation to generate adversarial examples which will misclassify the model.We utilize the strength of perturbations added by the misclassified examples to analyze the interpretability.Finally,the adversarial training is used to improve the robustness of the model,and it is further extended to the virtual adversarial training technology for semi-supervised training.The second method is based on attention mechanism that belongs to ante-hoc interpretability,which introduces the evaluation criteria of interpretability.This method first provides classification decisions for text classification tasks and generates fine-grained explanations.We associates explanation s with classification result to improve the quality of explanation.In order to generate a more reasonable explanation,we introduce the explainable factor as a criterion to evaluate the explanation.Next,jointly training the model through the minimum risk training method will make the classification better.Then we intuitively visualize the attention mechanism to show the impact of input on output,which will help people to understand the decision results.Finally,we explore the interpretability of the attention mechanism based on the results of the experiments.