The Research Of Deformable Image Generation Based On Generative Adversarial Networks

Images serve as one of the most important mediums for human beings to convey information.In recent years,the rapid growth of the computer vision community highly relies on the tremendous number of large datasets with immense amounts of annotated images.However,annotating images could be very cumbersome and costly.Therefore,the study on how to effectively generate images with natural annotations is drawing lots of attention and becoming more necessary.As a special kind of image generation,deformable image generation is super challenging because of the large structural deformation between the input and output.Nevertheless,its underlying vast application value and research value attract many researchers to work on it persistently.Considering the superior performance,generalization ability and flexibility of the Generative Adversarial Networks on the image generation task,we modify them to be applied to the Chinese calligraphy generation and pose transfer tasks.Based on the characteristics of these two tasks,we conduct the following researches:1)We propose to leverage generative adversarial networks to the task of Chinese calligraphy generation.The proposed model has two separate generators,called Supervise Network and Transfer Network.The former one reconstructs the calligraphy images while the latter one aims to transfer standard font images to calligraphy images.The essential function of the Supervise Network is to supply low-level supervision of its decoder part to that of the Transfer Network,in order to represent detailed local stroke information and global layout structure.2)We propose two novel mechanisms to the task of pose transfer,which are pose-attentional mechanism and progressive generation mechanism.The former one depends on pose features to construct attention masks which are used to selectively strengthen or suppress specific appearance features.The progressive generation mechanism helps to alleviate the difficulty brought by large deformation between the condition and target poses,resulting in more natural and realistic transferred person images.Owing to the advantages of these two mechanisms,our unified model exhibits obviously better appearance and shape consistency when compared to previous methods.Besides,our model contains far fewer parameters and achieves a much faster running speed.We also show our approach is able to augment the data of person re-identification datasets and considerably improve the person retrieval performance,especially under the cases of data insufficiency.Based on the above researches,we propose two effective deformable image generation methods on the basis of generative adversarial networks to handle Chinese calligraphy generation and pose transfer.Sufficient experimental results steadily verify the effectiveness of our proposed methods.