Research And Application Of Transformer-based Poetry Generation Model

The rapid development of artificial intelligence technology has provided a lot of convenience for people and gradually led to a new round of technological revolution.In this background,automatic poetry generation is of interest to many researchers,and Transformer,a neural network model based entirely on attention mechanism,has been widely used in the field of natural language processing because it has stronger knowledge transfer capability than the traditional recurrent neural network-based model.The research goal of this article is to apply the Transformer-based model to an automatic poetry generation task and develop a Chinese poetry generation system.Specifically,the main work and contributions of this thesis are:(1)Based on the way of working memory operation,a Transformer-based working memory poetry generation model TWM is proposed,in which a memory module is used to store the topic content and historical information of the poems,and a read-write module is used to guide the generation of the poems,in addition to the poem encoder and decoder using the Transformer model structure.Experiments show that TWM has a very significant improvement over the recurrent neural network-based model,with the model perplexity of 33.2 and 35.3 in the CCPC and CTP datasets,respectively,and also has very good results in BLEU and manual evaluation.(2)A topic and format controlled poetry generation model TFCPG is proposed.Firstly,a topic model is used to model the topic of the corpus,and then a multi-headed topic attention block is introduced in the decoder of Transformer to enhance the topic control when generating poems.Experiments on the CTP and CSP datasets show that the perplexity of TFCPG is 12.29 and 11.27,respectively,in addition,the topic hit rate and keyword hit rate are about 14% and 60% higher than Song Net,respectively.(3)A Chinese ancient poetry generation system based on the TFCPG model was developed.The system is a Web application with good cross-platform features and adopts a front-end and back-end separated development model,using a My SQL database and a Python-based Flask framework for the back end and a React framework based on the Type Script language for the front end.The reliability of the poetry generation function in this system was verified through experimental tests.