Multitask Text Classification Based On Deep Learning

Text classification is an important research task of text mining.Traditional text classification approaches represent text with bag-of-words features which fail to effectively capture word meaning and context information.Deep learning can solve this problem.Because it can not only effectively capture word meaning with word embedding,but also utilize word order to capture context information.However,the existing labeled dataset of text classification task is too small to fully train the large number of parameters in deep learning models.To solve this problem,we adopt the multi-task text classification technique based on deep learning for the text classification task.Because multi-task learning can utilize dataset of multiple related tasks to fully train deep learning models.After our investigation,we found that the existing models have two shortcomings.Firstly,the existing models do not consider the biased issue of the Recurrent Neural Network(RNN).They directly represent text with the output of RNN's last hidden layer,which lose much important information not in the end of text.Secondly,although the existing models adopt gating mechanism to help each task selectively accept shared features,the gating mechanism does not fully consider polysemy phenomenon,resulting in some irrelevant shared features being misused.To deal with the first shortcoming,we propose UP-RNN model that uses pooling layers to extract the important features of RNN's all hidden layer.However,the UP-RNN model only uses RNN to capture forward context features but ignores backward context features.Thus,we propose UP-BRNN model that uses a bidirectional recurrent neural network to capture the forward and backward features.However,the UP-BRNN model only captures shared features but ignores specific features.Therefore,we propose SP-BRNN model based on the gating mechanism.To handle the second shortcoming,we propose ASP-BRNN model with an Attention mechanism to identify relevant shared features for each task.In order to verify the proposed models,we conduct multiple experiments on four related text classification tasks.The experimental results show that: first,the proposed models can effectively solve the above shortcomings;second,our ASP-BRNN model outperforms the existing models on these four tasks.