Research On Extraction Summary Generation Technology Based On Attention Mechanism

The development of the Internet has made the group of Internet users continue to grow,and massive amounts of data information has been generated on the network and has exploded.You may have encountered such a scenario: when you click on the article link because you are interested in the title of the article,you find that the content of the article has nothing to do with the title.How to quickly and accurately obtain the text information we need from these massive amounts of information has very important research value.Text summary generation technology emerged as an important technology to solve this problem.With the deepening of research on information technology and artificial intelligence,the automatic text summary generation technology has become the mainstream method,but the automatic text summary generation technology methods still have many problems.In terms of pre-trained word embedding representation,how to more accurately represent the core content of a sentence has always been a problem that needs to be broken through in the field of natural language processing.From the perspective of the quality of the generated abstracts,there is generally a problem of sentence redundancy or semantic structure confusion between sentences and words.In terms of model generalization,the supervised model training method relies on manually written summaries.Therefore,the use of limited artificial summaries to train an abstract generation model for multiple fields is a key point for automatic summary generation technology to be widely used.In response to the above problems,this article mainly does the following three aspects of work:The article analyzes the decoder model commonly used in end-to-end models.Through analysis,we find that common decoders have information loss problems,which can be solved by adding attention models.Therefore,this paper first adopts Bidirectional Encoder Representation from Transformer(BERT)with better characterization ability as the encoding method of sentence vectors.In the encoder,the fine-tuning mechanism of BERT is used to extract fixed-length semantic vectors,combined with a Bi-directional Long-Short Memory Network(Bi-LSTM)makes the model more suitable for Chinese semantics and writing habits,and adds an Attention layer to the decoder to help the decoder to obtain the attention points that need attention at every time node.Finally,through a lot of experiments,it is proved that the extractive summary generation model of BERT's sentence-level coding method can eliminate sentence redundancy and improve the accuracy and readability of a summary.The addition of Attention mechanism makes the summary generated by the algorithm can restore the core content of the original text to a greater extent.In order to be more suitable for the manual writing of summary,the article proposes a twostage summary model based on Multi-Headed Attention based on the previous research results.Using the BERT word-level encoding as the input for the abstract generated in the previous article,and using the Multi-Head Attention(MHA)mechanism as the model decoder,the first-level summary generated in the middle is rewritten into the final summary,forming a two-segment summary generation model.Compared with the method of directly using the original text to generate the generative summary,the method of using the intermediate first-level summary has the advantage of the length suitable for the generative summary model,that is,the use of shorter content and more refined text provides the method of generative summary convenient.Experiments show that the short text word embedding coding of the two-segment summary model is more suitable for the generated summary model.The modified MHA mechanism can improve the readability of the generated summary,reduce the confusion of the sentence.Provide direction for the follow-up research work.Based on the proposed two-stage summary model based on the MHA mechanism,an automatic summary system with good interaction design and high summary quality is designed and implemented.Extensive system function test results show that the automatic summarization system can generate high-quality summaries with one-click text information on the network,providing practical applications for the algorithms proposed in the article.