Research On Text Automatic Summarization Combined With Transfer Learning

With the explosive growth of information data in the new era,people are facing the inevitable and challenging problem of information overload.This has raised interest in text abstraction techniques.The textual abstraction of large-scale text data in the era of big data is of great significance for people to quickly and accurately obtain valid data from massive data.Experience has shown that models trained in one text field often fail to achieve good results in another text field,ie the text automatic summary model is less robust.The automatic text summarization method requires a large amount of annotation data,ie,the original-reference summary pair,as the training set,but the existing annotation dataset covers a limited field,and it is difficult to train a good automatic summary in some areas where manual data is less.model.The work of this paper is to design a highly robust text automatic summary model and corresponding training methods to improve the versatility of text automatic summarization in various text fields.In this paper,the seq2 seq attention encoder-decoder automatic summary model based on the gated loop unit neural network is designed,and the model is optimized by combining the pointer mechanism and the coverage mechanism.In this paper,the gated loop unit is used to replace the cyclic neural network unit or the long and short-term memory network unit commonly used in the seq2 seq attention encoder-decoder,which can reduce the coding and decoding capability of the framework and significantly reduce the amount of parameters that the network needs to train,saving the number of parameters.Computing resources.Aiming at the problems of vocabulary and repetitive generation that are easy to appear in the generative abstract,this paper draws on the use of the pointer mechanism and the overlay mechanism proposed in the field of machine translation to reduce the occurrence of vocabulary and repetitive generation problems,so that the model generates automatic abstracts.The quality has reached an advanced level.This paper absorbs the idea of migration learning,and selects the fine-tuning method in model-based migration learning to improve.The characteristics of seq2 seq attention encoder-decoder automatic abstract model adopt differential fine-tuning,oblique triangle learning rate,connection pooling,and Layer thawing and other methods are applied to the training of pre-training models in the field of lack of labeled corpus,so that the automatic abstract model can obtain higher quality abstracts in the absence of field corpus,which effectively reduces the automatic abstract model of training texts.The amount of data dependency improves the robustness of the text automatic summary model.