Text Classification Based On Attention-Based LSTM Model

Text classification is a classical task of natural language processing research. Text classification contains several departments:data preprocessing, feature extraction and machine learning classifier training. In fields of image recognition and machine translation, deep learning has made great progress and is demonstrated has a great advantage in the data pre-processing and feature extraction. So we apply deep learning technology in the field of text classification. These technologies include Word Embedding, LSTM Model. In the meanwhile we use Attention Model to improve performance. The following are the main research work:(1) To solve the high dimensions problem and non-semantic feature problem of word vector, we use Word Embedding method to produce low-dimensional vector. These vectors also a characteristic that similar words have similar vector representation. We use these vectors as input of LSTM Model and this will improve the classification performance.(2) We first use LSTM Model for extracting text feature. LSTM Model solves the disappearance of the gradient problem and long-term dependence problem of RNN model by using 3 ’door’ to control. Then we use Attention Model to generate semantic code which contains probability distributions of input sequence’s attention. So this will reduce the loss of information and information redundancy.(3) The output of LSTM model only depend on the previous input and ignore the following input. So we combine the output of the Attention based LSTM model of positive sequence input and reverse sequence input as feature vector. Also we use Bi-LSTM model as a comparison model. So we can explore the impact of the context of text in text classification task.