The End-to-End Model Of Generating Poetry From Image

Poetry is the jewel of Chinese culture.There are countless poems in the history of the river,and people have been swayed by the wisdom of their predecessors and the beauty of poetry.Under the great wave of artificial intelligence,computer technologies such as image recognition,natural language processing,and subtitle challenge have developed rapidly,and the research on computer-generated poetry has made continuous breakthroughs.Most of the researches on the automatic generation of poetry in the past are in the generation of poetry from the inspiration of words,and very few scholars have studied the generation of poetry from images.At present,there are many problems in the research of image-inspired poetry generation,such as the lack of an end-to-end model for directly outputting poetry from image input,the inconsistency of output poetry content and image representation,and the poor quality of output poetry.The main work of this paper is as follows:(1)The model framework is based on an encoder-decoder,the encoder uses a full convolutional network FCN,and the decoder uses a long-and short-term memory network LSTM.Spatial visual features and semantic representations are extracted from pixel-level images using an FCN encoder.The FCN-LSTM based encoder-decoder model framework is used in the work of image-inspired poetry generation.(2)The model introduces the attention mechanism,and summarizes the characteristic information of all the outputs of the FCN encoder into a joint context summary layer through the fine-grained and semantic-guided attention mechanism,providing the decoder LSTM with information more conducive to poetry generation.(3)Introducing the memory model in the LSTM decoder,making the poetry output of this model more diversified and creative;constructing the image-poetry data set to prepare for the end-to-end model training of image-generated poetry.Three model evaluation experiments were designed: model structure analysis,poetry Turing test analysis,multi-model comparative analysis,and detailed analysis of experimental data.In the structural experiment,the PPL value of the confusion is significantly reduced after the introduction of the attention mechanism and the memory model,which proves that the quality of the poetry output based on this model is better.In the Turing test,approximately 44.8% of the poems in the non-professional group were considered to be machine-generated poetry or indistinguishable,proving that the poetry generated by this model is similar to the poetry level of human creation;in the multi-model comparison of poetry generation quality,The subjective and objective evaluation indicators of this model are excellent,compared with the traditional SMT model.The model is in the poetry evaluation of the five-speech poems and the seven-speech poems.The coherence of poetry is higher than the attention mechanism model ANMT.Out of 0.03,the meaning of poetry is higher by 0.17 and 0.09.