Research On Chinese Abstractive Text Summarization Based On Sequence To Sequence Model

With the fast growth of text information in recent years,massive text information is full of people's life,such as news,blogs,emails,conference reports and so on.Extracting main content from a large amount of text information has become an imperative requirement to solve the problem of information overload,while automatic text summarization provides an efficient solution.The main purpose of automatic text summarization is to condense the main content of the original document into a concise summary,which people can obtain document information effectively.In the field of automatic text summarization,most of the early work focuses on statistic extractive summarization and graph-based extractive summarization.In recent years,with the rapid development of big data and artificial intelligence technology,the automatic text summarization research is evolving from extractive summarization to abstractive summarization,which aims to generate higher quality summary.This paper mainly focuses on the abstractive summarization research based on sequence-to-sequence model.At present,it is still facing many challenges,such as the common problems of basic sequence-to-sequence model(word list overflow and generation repetition),lacking the ability of obtaining the main information of original document and generating summary inappropriately,etc.To solve these problems,this paper proposes corresponding solutions.The main research work and innovation of this paper are as follows:First,the subword method is adopted to deal with the problem of word list overflow.This method divides words into finer subword units,which reduces the length of vocabulary greatly and alleviates this problem.At the same time,an attention mechanism on output sequence is adopted to avoid repetitive contents.By reviewing the generated summary contents,this mechanism helps our model to weaken the redundant information contained in the current state and reduce the probability of generating duplicate content.Our model participated in the evaluation competition of NLPCC 2017 shared task3 and achieved the best ROUGE performance among all the participating teams.Second,in order to facilitate the summary writing,people usually sum up the main idea of the original document and mark out the topical words in advance when writing summary.However,most of sequence-to-sequence models allocate attention to all the contents of the original document rather than paying more attention to the important topical information,which increasing the difficulty of grasping the key information of the original document.In view of this,this paper proposes a new multiple attention sequence-to-sequence model which integrates topical keywords information attention mechanism.Specifically,firstly,we use the unsupervised method to identify the topical keywords of the original document.Secondly,this model combines topical keywords information with text semantic information to generate the final summary by a multiple attention mechanism.The evaluation results on the public Chinese single document summarization dataset of NLPCC 2017 shared task3 verify the effectiveness of this model.Finally,people usually revise immature summary several times to ensure the rationality of the summary.Nowadays,because most automatic text summarization systems cannot review and modify the generated summary,there are still many imperfections in the final summary.In order to solve this problem,this paper proposes a deliberate network model based on the latest global information.This model has two stages decoders.The second stage decoder combines the original document information with the updated summary information to deliberate on the immature summary generated by the first stage decoder,so as to ensure a higher quality summary.And the experiment results on the public Chinese single document summarization dataset of NLPCC 2017 shared task3 demonstrate the effectiveness of our model.