Text Classification Based On Deep Learning

As a fundamental research in natural language processing field,especially when the amount of the text data on Internet grows exponentially,the text classification has important research significance and extensive application prospect.In recent years,deep learning has gradually replaced traditional machine learning methods as the mainstream research direction in the field of text classification.The research in this paper mainly focuses on using the complex neural network architecture to simulate the way how the semantic is combined,this is using a variety of neural network architectures(mainly convolutional neural networks?recurrent neural networks and attention mechanism)from word representation to learn from layer to layer to get the final semantic representation of the text.Considering the fact that the advantages of recurrent neural networks and convolutional neural networks are respectively extracting global features(like long term dependency)and local features,a novel network architecture,BLSTM-Inception,which has combined the bidirectional LSTM and Inception modules is proposed in this paper.The forward and backward hidden states,respectively coming from the forward and backward LSTM,are concatenated as double channels rather than directly added as single channel which can avoid the information loss.BLSTM-Inception v1 uses the global max pooling to extract the text representation which can effectively avoid the interference from lots of noise features in the extracted feature maps and significantly reduce the number of parameters in the softmax layer.Extensive experiments conducted on five text classification tasks in this paper show that the propose BLSTM-Inception v1 model outperforms the state-of-the-art deep network model proposed in the past two years.Because word must be combined with context to form phrase to have clear semantics,so the text representation generated by use attention mechanism to learn the weights of phrases is more precise than learn the weights of words.Therefore,a novel neural network framework based on phrase-level attention mechanism,NN-PA,is proposed in this paper.Compared with the bidirectional LSTM architecture based on word-level attention mechanism,the major improvement of NN-PA is to add convolutional operations after word embedding layer to extract the representations of local phrases.In addition,five kinds of attention mechanisms have been tried in this paper,three of which are modified by the attention mechanisms used in neural machine translation field.Extensive experiments conducted on five text classification tasks in this paper show that the NN-PA models based the first two attention mechanism outperform the state-of-the-art deep network model proposed in the past two years.