Research And Application Of Abstractive Text Summarization Based On Deep Learning

As an important branch of natural language processing,automatic text summary has attracted the attention and investment of many researchers.The research of automatic text summary started in1950 s and has gone through several stages of development.The initial research is mainly based on the statistical method,which selects and sorts sentences according to the characteristics of word frequency,position,length,etc.,and generates extracts.Later studies introduced linguistic knowledge,using syntactic,semantic,textual analysis to understand and reconstruct the text,to generate abstracts.In recent years,with the development of deep learning and other technologies,automatic text summary research has entered a new stage,using neural network models to process text and generate more smooth and natural summaries.Automatic summarization systems can be modeled in two ways,namely extraction and generation.When modeling using extraction techniques,the main part of the text is to extract important sentences from the source document according to some scoring criteria and then concatenate them together to form a summary.The generation-based approach is more complex and challenging,requiring that the model first understand the meaning of the source document and then generate a short summary based on what it understands.This paper mainly studies the generated text summary based on deep learning.The specific work of this paper is as follows:First,due to the problem of exposure bias caused by Teacher-Forcing mechanism training sequences generating models,existing efforts attempt to improve by modifying training data to simulate the results generated by models.However,the current work does not consider the semantic association between the original training data and the modified training data.This paper proposes a generative text summary model based on semantic regularization.The scheme achieves regularization by measuring the semantic similarity between the original sample and the modified sample.Specifically,the source document is first input into the encoder to obtain the semantic representation of the source document,and the token in the reference abstract is randomly masked to obtain the modified sample,the reference abstract and the enhanced sample are respectively generated by the decoder,and the generated representation and semantic representation are obtained by cross-attention to obtain the respective semantic vector.Finally,the cosine similarity is used to obtain the semantic similarity and the semantic regularization loss.The experimental results show that the scheme proposed in this paper can alleviate the exposure bias problem to a certain extent and obtain better results compared with the scheme of modifying training data.Secondly,considering the inconsistency of optimization objectives and evaluation indexes in sequence-to-sequence models widely used in generated text summaries,this paper proposes a comparative learning method based on online sampling.Firstly,the model was used as a generative model to fine-adjust the maximum likelihood estimation(MLE)loss.Then,the model was used as a reordering device to sort the candidate abstracts generated by the model to get the contrast loss.The multi-task learning method was used to optimize the model by combining MLE loss and contrast loss.The main idea of this method is to ensure the accuracy of word-level prediction through MLE loss,and to make the model assign higher generation probability to the candidate abstracts with higher evaluation index scores through comparison loss.In addition,in the stage of comparative learning,this paper uses online sampling to generate candidate abstracts,so as to ensure the consistency of the model as a generation model and the sequencer.At the same time,the quality of candidate abstracts can be constantly improved to ensure the learnability of the sorting task objectives.The experimental results show that compared with the basic sequence-to-sequence model and off-line comparison learning scheme,the proposed scheme can obtain the optimal results in Xsum dataset for the valence index ROUGE.Finally,this paper experiments the prototype of the generated text summary system,and carries on the related demand analysis,system design,the above scheme is applied in it,and finally realizes and displays the system.