Abstractive Text Summarization Generation Method Based On Adaptive Resilient Loss

In recent years,abstractive summarization models are preferred over extractive summarization models as they can generate words that do not exist in the original text,whose summary descriptions are more flexible and natural.The sequence-to-sequencebased abstractive text summarization model learns the pattern of summary generation from the training data by modeling the relationship between the original text and the reference summary.Although abstractive text summarization based on sequence-to-sequence model has the advantage of being able to generate text freely,due to the high uncertainty of both the training data and the existing models,the effect of the summary generation of existing methods is still poor.The first point is the characteristic that the sequence-tosequence model is highly dependent on the reference summaries and the uncertainty of the training data,the combined effect of the two may bring losses to the performance of the model.The second point is that the softmax output commonly used in sequence-tosequence models has a long tail effect,which reduces the probability to output correct words,making it easy for the model to generate rigid or repetitive summaries.To solve the above problems,the paper designs Loss Mask and Adaptive Sparseness method,by studying the uncertainty of data and model in the training process.The Loss Mask method uses the abstraction degree of the reference summaries to weight the loss,so as to selectively learn from training data.Adaptive Sparseness uses sparsemax to sample the output probability of the model,so as to output summary content with high certainty.The paper applies these two methods to the Pointer-Genertor Networks and Transformer models.The experimental results on CNN-Daily Mail and LCSTS datasets show that these two methods improve the ROUGE score of the generated summaries.