Research On Text Summarization Technology Based On Improved Transformer Model

With the rapid development of the Internet era,information has exploded.The speed at which people read and process information can no longer keep up with the growth rate of information.How to quickly and efficiently extract important information from massive amounts of information and apply it to actual scenarios has become a problem that needs to be addressed urgently.Automatic text summarization technology is a method of generating sentences that highly summarize the source text.This technology can effectively improve the information overload situation.At present,automatic text summarizati on technology is mostly based on the encoder-decoder structure.The input sequence of the encoder has a length limitation.If the input sequence is too long,the too long part will usually be deleted,which will cause the input sequence to lose part of the original information.RNN and its variants are the mainstream methods for building encoders,but they are sequential,difficult to process text sequences in parallel,and training efficiency is not high.At the same time,the abstract generated by this framework may have many repetitive sequences and OOV words(not appearing in the vocabulary).In response to the above problems,this paper proposes a two-stage framework that integrates extractive and abstract ive text summarization techniques to complete the task of summary generation.Experiments on two different types of datasets,CNN/Daily Mail and Wiki How show that the framework is suitable for text summarization problems.The main research contents of this paper are as follows:1)We design a supervised sentence ranking model for articles with titles.We build encoders based on the self-attention mechanism,extract the matching features of the sentence and the title from different levels,and calculate the possibility that the sentence can be used to generate a summary.The sentence of the article is rearranged according to the probability,and the input text is provided for the summary generation model.At the same time,the vector obtained by encoding the sentence is retained as the embedding vector of the input word of the summary generation model.2)We design an unsupervised sentence ranking method.The sentence vector is obtained by the weighted summation of the word vectors of the words in the sentence,and the weighted graph is constructed based on the semantic similarity of the sentence vector.We use the semantic level Text Rank algorithm to rank the sentences in the article.3)We study the task of summary generation based on the Transformer model,use the time penalty mechanism to improve the attention mechanism that connects the encoder and the decoder,and use the pointer network to control the summary generation model in the last layer of the decoder to extract words from the original article as candidate abstract words with a ce rtain probability.