| In recent years,the rapid development of Internet technology has brought great convenience to people’s daily life,but it has also inevitably led to a rapid growth of information,and it has become particularly important to obtain the required information quickly and efficiently under such circumstances.The emergence of automatic text summarization technology can effectively alleviate this problem.As one of the important research contents in the field of natural language processing and artificial intelligence,it uses computers to automatically distill a short concise and coherent paragraph from long texts or text collections that accurately reflect the central content of the source text,with algorithms including feature scoring,classification algorithms,linear programming,submodular functions,graph sorting,sequence annotation,heuristic algorithms,deep learning,etc.In this paper,we propose a lightweight automatic text summarization model by delving into knowledge related to datasets and evaluation metrics,and predict possible future challenges and trends.Automatic text summarization tasks can be divided into two approaches: in extractive summarization approaches,the model generates a summary by selecting meaningful sentences from the source text;in generative summarization approaches,the model generates a summary by encoding the source text and using machine learning.Both of these approaches have been studied and achieved relatively good results in various forms on text summarization tasks,including graph-based and deep learning-based approaches.The use of large-scale pre-trained language models in natural language processing tasks is becoming increasingly common,but running these voluminous models remains challenging in the face of insufficient computing power and limited computing resources.There are 3 innovative points in this article:(1)To address the problems of traditional text summarization models with many parameters and time-consuming operations,this paper explores the summarization performance exhibited by DistilBERT,a distillation variant of the BERT model,on CNN/DM datasets based on the improved BERTSUM model,and thus proposes a lightweight extractive summarization model--DistilSum.(2)For the work of model lightweight,this paper proposes another lightweight extractive summarization model-MobileSum,relying on knowledge migration and using teacher-student networks.this model is more suitable for application on low-resource devices such as cell phones,and compared with the DistilSum model,it has fewer model parameters and faster training faster than the DistilSum model.(3)In order to improve the model performance,this paper improves and introduces structured attention in the model,which is used jointly with the summary judgment layer of the model to score alternative sentences and finally helps the model to select the optimal top-n sentences as document summaries.After experimental comparison and analysis,DistilSum model guarantees 99.9% of the performance of the original model,while reducing about 36%of the training parameters,which significantly reduces the training time.mobileSum model guarantees 94% of the performance of the original model,while reducing 79% of the model parameters,which successfully achieves the lightweight of the model. |
