Research On Text Classification Based On Deep Learning

Text classification is a fundamental task in natural language process-ing(NLP),which is widely used in various applications such as spam detection,sentiment analysis and topic classification.Recently,neural networks have achieved remarkable results in natural language process-ing.This paper aims to study the text classification tasks based on neural networks from three different perspectives including novel methods or ar-chitectures.The main content of the thesis are as follows:1.For text classification,traditional local feature driven models learn long dependency by deeply stacking or hybrid modeling.This paper pro-poses a novel Encoder l-Encoder2 architecture,where global information is incorporated into the procedure of local feature extraction from scratch.In particular,Encoderl serves as a global information provider,while En-coder2 performs as a local feature extractor and is directly fed into the classifier.Meanwhile,two modes are also designed for their interactions.Thanks to the awareness of global information,our method is able to learn better instance specific local features and thus avoids complicated upper operations.Experiments conducted on eight benchmark datasets demon-strate that our proposed architecture promotes local feature driven models by a substantial margin and outperforms the previous best models in the fully-supervised setting.2.As neural networks have dominated the state-of-the-art results in a wide range of NLP tasks,it attracts considerable attention to improve the performance of neural models by integrating symbolic knowledge.Differ-ent from existing works,this paper investigates the combination of these two powerful paradigms from the knowledge-driven side.We propose Neu-ral Rule Engine(NRE),which can learn knowledge explicitly from logic rules and then generalize them implicitly with neural networks.NRE is implemented with neural module networks in which each module repre-sents an action of a logic rule.Besides,the types and orders of actions are determined by a neural parser or a symbolic parser.The experiments show that NRE could greatly improve the generalization abilities of logic rules with a significant increase in recall.Meanwhile,the precision is still maintained at a high level.3.Dropout is used to avoid overfitting by randomly dropping units from the neural networks during training.Inspired by dropout,this pa-per presents GI-Dropout,a novel dropout method integrating with global information to improve neural networks for text classification.Unlike the traditional dropout method in which the units are dropped randomly ac-cording to the same probability,we aim to use explicit instructions based on global information of the dataset to guide the training process.With GI-Dropout,the model is supposed to pay more attention to inapparent features or patterns.Experiments demonstrate the effectiveness of the dropout with global information on seven text classification tasks,includ-ing sentiment analysis and topic classification.