Multi-Label Text Classfication Algorithm Based On Seq2Seq Model

Multi-label text classification is a method to assign multiple labels to each text by analyzing their semantics.With the success of seq2seq model in machine translation,style transfer,abstractive summarization,and other fields,the model can be used to generate positive labels corresponding to each text.However,there will exist two problems including exposure bias and the need to predefine the label order when the seq2seq model is utilized to solve the multi-label classification problem.Additionally,multi-label learning also needs to pay attention to the correlation among labels and the relationship between the instance and the label.In order to solve these problems,this thesis proposes a seq2seq model based on graph embedding and region attention mechanism.Moreover,a seq2seq model based on joint embedding and multi-output is proposed to solve the exposure bias and improve the overall performance of the model.The main research achievements of this thesis are as follows:1.The seq2seq model consists of encoder,attention mechanism and decoder,this thesis makes some improvements to make the Seq2 Seq model suitable for multi-label text classification as follows: Firstly,the graph embedding technology which can mine the associated information among labels is applied to generate the label vector for the encoder.Furthermore,unlike machine translation task,the model needs to pay attention to the information of multi-region of the text when predicting the current label in multi-label text classification task.Based on this problem,this thesis improves the dot attention mechanism commonly used in machine translation,and designs the region attention mechanism which focuses on the information of text segments.Finally,the policy gradient is utilized to solve the problem of exposure bias and predefined label order.2.This thesis proposes a seq2seq model based on joint embedding and multi-output.First of all,the joint embedding strategy utilities the label information of K-Maximum probabilities at the previous timestep to predict the current label,and combines scheduled sampling strategy to effectively solve the exposure bias.Finally,the generalization performance of the model can be improved effectively by training multiple decoders from multiple ways and merging results of multiple decoders at inference stage.Experiments on two multi-label text datasets show that graph embedding and region attention mechanism improve the performance of the model in chapter three,the generalization performance of the model in chapter four is improved by using joint embedding and merging multi-output,furthermore,the performance of proposed models are superior to the state-of-the-art methods on two main metrics.