| With the rapid development of digital media and network technology,the cartoon industry has become one of the booming industries.As a unique form of artistic expression,comics can not only bring joy and enlightenment to readers,but also convey profound thoughts and feelings.In the process of comic creation,you need to sketch the cartoon first,then add color and texture to it.However,in the past,this process usually required comic creators to complete purely by hand,consuming a lot of time and energy.In recent years,with the continuous development of generative antagonism network in the field of image generation,researchers begin to explore the method of fully automatic cartoon sketch to image generation,that is,the method of reasoning color texture according to cartoon sketch to make it into cartoon image,including unguided method and guided method.There are also some comic enthusiasts who are interested in the classic black and white collection of comics,hoping to create the collection of comics again through automatic image generation technology.But because the collection only has paper versions,there is no digital version available,so it cannot be automatically colored.Aiming at the above problems,this paper studies three aspects,namely,data set construction of comic sketches in collection,unguided comic sketches to image generation method,and comic sketches with reference drawings to image generation method.The main research contents are as follows:In this thesis,three aspects are investigated to address the above issues,namely,the method of constructing comic sketch dataset in the collection,the method of unguided comic sketch-to-image generation and the method of introducing reference map for comic sketch-to-image generation,which are mainly studied as follows:(1)A crowdsourced comic sketch data set construction method is proposed,which can scan the comics in the collection into digital images,clean and label the digital images through an interactive online annotation platform,and finally build a large-scale comic sketch data set Comic Lib.The data set contains 181,354 cartoon sketches and 2107,648 annotated objects scanned from the comic book library,which can be used to complete tasks in sketch related fields,such as sketch recognition,sketch target detection,sketch retrieval and sketch to image generation.This paper verifies that Comic Lib has more advantages than other sketch data sets through multiple experiments.(2)In the study of unguided cartoon sketch to image generation method,the Pix2 Pix model is improved to solve the problems such as blurred edges and uneven color transitions of Pix2 Pix image generation.The Res Net Xt residual structure was introduced into the generator,and a new generator structure Res Xt UNet was proposed,which was used to transfer sketch features more effectively and alleviate the edge blur problem.Perceptual loss is added to the loss function to solve the problem of uneven color transition in Pix2 Pix images.The rationality and effectiveness of the two improved methods were verified by ablation experiment and comparison experiment.(3)A self-supervised learning model Hint GAN is proposed to solve the problem that real images cannot be input into generators for training as reference images in the study of the method of image generation by introducing comic sketch of reference images.The fuzzy real image is used as a reference image to form a group of images that can be used for training with anime sketches and real images.The fuzzy processing method can effectively reduce the interference of reference contour structure to the generated image.Hint GAN adds a style extraction network on the basis of Res Xt UNet,which is used to extract the features of the reference map,and fuses the style information with the sketch information through the Ada IN method to generate an image containing the reference map style and sketch outline structure.By comparing with existing methods,HintGAN has better image generation capability. |
PDF Full Text Request