Research And Implementation Of A Strong Abstractive Summarization System

Automatic summarization is a key natural language processing technology for mining the value of massive contents and filtering redundant information.Among many summarization researches,abstractive summarization has attracted much attention from researchers and engineers because of its excellent performance on conclusion,coherence,and closeness to human summarization.However,the current mainstream abstractive summarization models,which are built based on the sequence-to-sequence deep model for solving machine translation tasks,tend to degenerate into extractive-like summarization models,which means the generated summaries will copy the sentences from input articles word by word.Such degenerated models are defined as weak abstractive summarization models in this thesis,as opposed to strong abstractive summarization models with higher abstraction.This thesis explores how to solve the degeneration problem of abstractive summarization models and builds a strong abstractive summarization system from three perspectives.From the perspective of data,we find that there exists subjective bias in the large-scale abstractive summarization dataset,and we extract the subjective style embedding of samples by designing a self-supervised summary ranking task,and then further cluster the embeddings to eliminate the subjective bias to obtain higher quality data.From the perspective of the model,we design a diverse attention regularization based on the determinantal point process,which considers both the quality and diversity of attention,and prevents the learned attention from degeneration.It improves the abstraction ability of the model.From the perspective of the task,we explore the effect of endto-end summarization task by splitting the document-level summarization into a summary extraction task and a sentence-level rewriting task.By control ling the summary compression ratio,we study the impact of the task setting on the abstraction ability of the model.The experimental results show that both high-quality data and welldesigned regularization can improve the abstraction level of the modelgenerated summaries,which is the first step towards a strong abstractive summarization system.Experiments that split document-level summarization into two tasks and explore the impact of different task settings on model training also provide some evidence that inappropriate compression ratios tend to make system degenerate.An end-to-end document-level abstractive summarization with larger compression ratios is not the most desirable task setting for the current model architecture.In addition,this paper also designed a strong abstractive summarization experiment system,including two major functions of model interaction and summary analysis.This system can analyze and improve the strong abstractive summarization model intuitively and quickly.