Improved Attentional Seq2seq With Policy Gradient For Text Summarization

With the explosion of digital information,text summarization has flooded in our lives nowadays,and it is helpful to alleviate the problem of information overload.The main techniques of text summarization contain compressive,extractive and abstractive summary.Among them,the extractive summary is composed of existing sentences extracted from the original text.The compressive summary is formed by extracting and simplifying the important sentences from the original text,and then combine them in a natural order to make the summary.The abstractive summary is formed by rewriting or reorganizing the original text.Obviously the compressive summary is similar to the extractive summary and both of them are relatively simple,but they can not generate the new words differ from the original text,so it is difficult to guarantee the consistency between the summary and the original text.while the abstractive summary is more near to the logical thinking of human beings.With the rapid development of deep learning technology,the abstractive summary has made a great progress.Some common technologies in abstractive summary are also familiar to us,like word embedding,word2vec[1](word to vector),one hot representation(one hot encoding),seq2seq(sequence to sequence[2])and so on.Abstractive summarization has been relatively mature,but the quality of generated summary is far from perfect.For example,if the paragraph is too long,the machine will take a long time to understand the paragraph,and the machine will lose the memory of the paragraph information.And deep learning heavily relies on labeled data,labeling for text is also a time-consuming task.What is more,other problems like OOV[3](out of vocabulary),wrong predicted words,unaligned problem between the ouput target words and input source words,these problems are all need to solve by a more advance model in the future.To solve these problems in my research,I propose the improved attentional seq2seq model with policy gradient method to solve the shortcomings in the basic sequence to sequence model(Encoder-Decoder structure[4])for text summarization.I do the mainly improvements as below:1 Use soft attention mechanism[5]in decoder to solve the unaligned problem between the output target words and the input source words.I use the convolution kernel function conv2d to realize the multiply operation.The attention mechanism can be realized by putting attention masks on hidden layer.2 Add the reinforcement learning method:policy gradient[6]to make targeted optimization for evaluation indicators(ROUGE-1,ROUGE-2 and ROUGE-L).Specifically speaking,I use the Policy Gradient method to update the weights in network when training.I take this training part as the actor,and take the scores of ROUGE for the generated summary as the critic.That is to say,when I update the weights in network,if it will lead to a better scores of ROUGE for generating summary,I should encourage the tend of changing the weights in network.Otherwise,I need to punish the tend of changing the weights.3 I use some tricks during my experiment:use word embedding method[7]to reduce the amount of data required for training;Add drop out in LSTM cells[8]of encoder to solve the overfitting problem[9];Use scheduled sampling[10]in decoder to solve the inconsistency problem between training and testing step;Use mini-batch gradient descent method[11]in training to solve the unconstrained optimization problem;Use beam search algorithm[12]when generating the summary,it can greatly improve the searching efficiency when the searching space is very large.After my modifications,the attentional seq2seq with policy gradient method for text summarization can get a better scores as:0.3698 in ROUGE-1,0.1598 in ROUGE-2,0.3380 in ROUGE L.