Research On Long Text Abstract Generation Based On Deep Learning

With the society entering the information age,the data on the Internet is growing explosively.The amount of data is not only huge,but also the data dimension is too high.It is very important to effectively solve the information overload and mine useful information from the massive data.Most of the network data exists in the form of text,so the technology of text summarization is the key to get value information from a large number of text information.In recent years,generative summarization technology is becoming more and more mature,especially the automatic summarization technology based on sequence to sequence architecture combined with attention mechanism framework,which makes the generated summarization achieve good results,but there are many shortcomings in the face of long text.First,the coding end uses the word vector input form of single feature,which leads to the lack of text feature mining,and then affects the neural network The model extracts the text semantic information.Second,in the coding stage,the information obtained by the two-way cyclic neural network is simply spliced,the text sequence is too long will produce the problem of distance dependence,which can not effectively mine the potential semantic information of the context,and the generated summary often lacks or deviates from the core information of the original document.Third,in the decoding stage,using the attention mechanism and the decoding form of neural network,the generated abstract has the problems of semantic incoherence and semantic lack,which results in the low accuracy of abstract generation.In view of the above problems,this paper proposes a long text summarization generation model based on deep learning,mainly from three aspects: feature words to quantization,encoder and decoder1.The encoder based on deep communication agent is constructed.In this paper,the frequency and semantic features of words are introduced,and the TF-IDF value of feature words and the semantic value obtained by LSA are integrated to form a new word vector through naive Bayes formula.It effectively integrates the multi-dimensional features of the text,improves the ability of word meaning understanding,and helps the encoder learn the text information.In view of the lack of long text coding ability and the inaccuracy of intermediate semantic acquisition of traditional cyclic neural network,a deep proxy communication mechanism isintroduced.The encoder with this structure can obtain global information more accurately and improve the ability of text understanding.2.A two-stage summary generation model is constructed.In the decoding end,we use two-stage decoding structure,the attention mechanism and the one-way neural network LSTM to generate the intermediate text semantics,propose the multi-agent pointer network to calculate the probability distribution of the decoding vocabulary,generate the abstract draft,use BERT to refine the abstract draft sequence,improve the coherence of the abstract semantics,introduce the multi-attention mechanism,improve the memory ability of the decode to the source document sequence,and solve the problem of word repetition in abstract can generate a more natural summary sequence and improve the quality of summary generation.3.Experiment.The validity of word feature integration is verified on SnLi dataset.Evaluated Summary generation model on CNN / Daily Mail data set,and selected four indexes of Rouge-1,Rouge-2,Rougr-l and Rouge-avg to evaluate the summary quality,In addition,the generated summary is evaluated manually,The experimental results show that the summary generated by this method has higher accuracy.