Research And Implementation Of Automatic Text Summarization

With the advent of big data information times,people increasingly rely on the network day by day.Because the network information is very complex,people need to extract critical information from numerous pages.The length of Web information is long,and the distribution of crucial information is uneven.Hence,it takes people much time to browse web content completely.During the detection process,because the detection standards are distributed in each chapter evenly,and it is necessary to read multiple chapters to extract and summarize.The researchers need to consult a large number of documents to find relevant knowledge.At present,the model mostly deals with the short text datasets with obvious summary such as news.For the text content with even topic information and long length in the source text,the obtained summary often has some shortcomings such as it is difficult to extract the key information which on the later place,inaccurately reproducing factual details,an inability to deal with out-of-vocabulary(OOV)words,and repeating themselves.The purpose of this paper is to obtain the summary from the evenly distributed text information and apply the algorithm to the prototype system.We proposed an algorithm framework combining generation and extraction to improve the summary generation algorithm.Our main innovations are as follows.The improvement of extraction and generation algorithms based on the pre-training model is proposed to enhance its global information memory.Combining the advantages of the two algorithms,a new joint model is proposed to make the text information evenly distributed,and the generated summay is more consistent with the original information,which reduces the repetition rate.In order to make the model better deal with the original information with relatively even distribution,this paper proposes a new extractive algorithm model and a generative algorithm model,and a summary generation model that combines the extractive and generative algorithms on multiple datasets.A comparative experiment was conducted,and a small,uniformly distributed private data set was constructed.In a number of comparative experiments,the evaluation index has been increased by a maximum of 1.8 percentage points,proving the effectiveness of the method,and building a summay automatic generation prototype system to display the results.