Research On Image Translation Method Based On Adaptive Multiscale Information Fusion

In the field of computer vision,image translation includes a variety of image generation tasks such as image coloring.In the process of translation,the source image is translated into a new image through encoder coding and generator decoding,so as to realize the editing of source image,which is of great significance in practical application.Since the great difference between the source image and the target image,the processing ability of the image translation model directly affects the quality of the generated image.This thesis improves the processing ability of the model by improving the generation structure of the image translation model to obtain higher quality images,and further expands based on this to achieve controllable and editable diversified image translation.Most of the existing image translation models are based on conditional generative adversarial network,in which the generator generates a specified size image through continuous up-sampling.The simple generation structure can not get the effective information of the image completely,which causes the generated image to produce artifacts or lose details.In order to solve the above problems,inspired by multi-scale information fusion in image processing tasks such as image classification,this thesis uses multi-scale information to improve the generation structure in the generator,and then proposes an image translation model SK-GAN(Selective Kernel Generative Adversarial Network)based on multi-scale information fusion.SK-GAN uses SKBlock(Selective Kernel Block)to construct a multi-scale information fusion module Res-SKBlock(Residual SKBlock)based on residual structure,which obtains and fuses multi-scale information at each up-sampling stage of the generator,and then transmits the information to the next up-sampling stage by channel level adaptive fusion.In this manner improves the generation structure of the traditional translation model,and controls the translation of different scale information by means of the adaptive fusion,so that the generator can obtain the dynamic receptive field and improve the quality of the generated image.In addition,this thesis also adds Res-SKBlock in the up-sampling stage of Cycle GAN to further verify the performance of ResSKBlock.The experiments show that SK-GAN can obtain high-quality images on multiple datasets by the Res-SKBlock improved generator,and Res-SKBlock not only plays a good role in SK-GAN,but also helps to improve Cycle GAN.The diversity of generated images is restricted in traditional translation model since the input of single source image,which results in the model producing certain output and unable to perform deeper translation on the same source image.In order to make the translation model produce diversified results for the source image,this thesis proposes a dual branch guided image encoder to enhance the diversity generation ability of the translation model,and based on SK-GAN designs a translation model GSK-GAN(Guided SK-GAN)with guided image in the task of sketch synthesis realistic image,which generates a corresponding style of generated image according to the information of the guided image such as color and texture.GSK-GAN uses the dual branch guided image encoder to extract the feature of the guided image,and transmits the feature information to the generator by the parameter generator and feature transformation layer.Compared with the existing way of bi-directional information transmission between different encoder,the guided image information fusion method proposed in this thesis not only guarantees the image quality,but also is more conducive to the expansion of the translation model.The traditional guided image encoder can make the translation model use the guided image to generate a specified style image,but the change of image style is still limited to guided image.The dual branch guided image encoder proposed in this thesis controls the translation degree of feature information through the weight corresponding to each branch,so as to achieve interpolation between the generated images of different guided image corresponding styles,and reduce the restriction of the guided image on the image style.In addition,the dual branch guided image encoder is also used to learn the latent variable distribution of the guided image to get more random styles,and further enhance the diversity of GSK-GAN generated images.The experiments show that GSK-GAN can not only generate reasonable images in the process of controllable style translation,but also obtain more styles by interpolation and latent variables.