| With the rapid development of Internet technology and smart communication devices,people are exposed to more and more fragmented data information.In real life,different data information is often stored and used in different modal forms.Therefore,how to use different modal data to accomplish specific tasks has become a hot issue for research in the field of artificial intelligence at this stage.Cross-modal generation is an important research direction in cross-modal intelligence,and its development is important for improving the interaction between cross-media data and promoting the analysis of multi-modal data.As a sub-task of cross-modal generation,text-generated image refers to a model that can generate high-definition,realistic images based on the semantic understanding of the input text,which has broad application prospects in art creation and virtual reality.In recent years,with the rapid development of machine learning technology,images generated based on text have made breakthroughs in terms of generation quality and visual realism.However,traditional machine learning-based text-generated image methods still face great challenges in terms of the diversity and semantic consistency of the generated images.Therefore,this thesis improves the diversity and semantic consistency of the generated images by mining the modal correlation between the semantic and visual information of text.The details of the research are as follows.(1)A semantic consistency-based adversarial network image generation method(GAN-SC)is proposed,which deeply explores the semantic association information between multi-modal data and achieves efficient cross-modal generation by fusing semantic parsing and image generation techniques.The whole generation network model is divided into image generation module and semantic comparison module.In the image generation module,a mixed attention mechanism is designed to guide the generator to pay attention to the image details at different stages,so as to improve the visual authenticity of the generated images.In the semantic comparison module,a semantic comparison scheme is designed to map multi-modal semantic features to the same semantic space for feature comparison,and the semantic difference between text and image is reduced by minimizing consistency antagonism and classification loss function.In addition,numerous experiments were conducted on the CUB bird and MS coco datasets to evaluate the performance of the method.Experimental results show that the proposed method is superior to traditional methods in semantic fidelity and visual reality.(2)A Siamese Network image generation method(SGNet)based on attention mechanism is proposed,which combines the Siamese network,attention mechanism and multi-level discriminator to generate semantically consistent and visually realistic high resolution images based on the analysis of semantic information of text description.This method uses the multi-stage generation network to construct the basic framework of semantic information acquisition and image generation methods,and introduces spatial and channel attention mechanisms to capture the deep semantic correlation between text and image.A parallel Siamese network method is designed to reduce the feature difference between two text image pairs and generate semantically true images.In addition,a two-path multi-layer semantic alignment scheme is proposed to further optimize the generated images.After training on the CUB bird datasets and MS coco datasets,the method was fully tested and evaluated for generation.A lot of comparative experiments and ablation analysis show that the proposed method has better performance in semantic optimization and image generation. |
