Research On Text Summarization Method Based On LSTM Sequence To Sequence Model

In the era of big data,blog,news articles,reports and other text information on the Internet are growing unprecedentedly.It is a challenging task to retrieve useful information from a large number of texts.Automatic text summarization technology provides an effective solution for summarizing the important information of these texts.With a short summary,the text content can be effectively retrieved,processed and digested.At present,most popular machine learning methods use the sequence to sequence architecture based on recurrent neural network(RNN),which has many problems,such as not well retaining the important vocabulary and syntax structure of the source text,generating a large semantic difference between the summarization and the source text,and generating some redundant information.In order to solve the above problems,this thesis research the method of text summarization based on LSTM sequence to sequence model,designs and implements an improved encoder-decoder architecture to help the model generate text summarization with more topic information,less redundant information and more accurate syntax structure.Specifically,the main contributions of this thesis include the following parts:Firstly,a improved encoder structure is designed and implemented in this thesis.Before input to the encoder,the source text sequence is pre processed and then replaced by the word embedding of word2 vec model training.Then the ELMo model,which can understand polysemant according to the context,is used to replace the former to improve the word embedding method.The encoder uses the bidirectional LSTM neural network and introduces the topic fusion attention mechanism at the same time.The LDA topic model is used to extract the potential topic information of the text,expect to bring the prior knowledge to the text summarization;On the basis of this improved encoder structure,this thesis introduces a copy mechanism to enhance the sentence structure information of the summarization.By adding the part of speech and interdependency syntactic relationship to the model,in order to avoid missing the words that play a key role in the sentence structure of the source text and the dependency between words.Through its various combinations in the encoder structure,to compare the impact of different combinations on the results of summarization generation.Secondly,the LSTM neural network is used to the decoder,the coverage mechanism and the coverage based regularizer are introduced in the process of summarization to carry out additional coverage training on the model and select the model with the lowest verification loss.The improved beam search method is introduced to reduce the search space of summary words to improve the search efficiency.In the beem search algorithm,a reward and punishment mechanism is added to keep the summarization highly related to the source text.The effectiveness of these techniques in text summarization was studied through a series of controlled experiments.Finally,the improved encoder-decoder is tested on LCSTS public data sets,and the ROUGE index is used as the evaluation standard.The experimental results show that the improved model contributes to promising improvement in the task of automatic text summarization.