A Study On Image Style Transfer Based On Generative Adversarial Network

Image style transfer is a technology that changes the image effect by processing the color,contour,line and other information of the image with computer technology.Traditional style transfer required the use of a dedicated style model,while the subsequent emergence of a generic framework to handle multiple style transfer has the limitation of matched-data-pair requirement.As a new general framework of multi-style transfer,Cycle GAN,which is an improved version of original GAN with the adding cycle-consistency loss,has the benefits of paired data sets removing,thus is more promising.However,because the training of the algorithm is unsupervised,there will be background color distortion and insufficient conversion exists.In this paper,the Cycle GAN-based image style transfer is applied to the artistic style transfer of real images,and some of the existing problems are studied.By using a generator with Skip-Res Net architecture and the nearest neighbor interpolation to replace deconvolution operation,the problems of background color distortion and insufficient conversion are resolved.Firstly,on the basis of comparing the quality of samples generated by different objective functions and the difference of their training success rate,the least square distance was selected as the objective function.Secondly,Skip connection is added to the residual module of Cycle GAN generation network to form a generator with Skip-Res Net structure,of which the utilization of original image features in the conversion process is enhanced.Then the operation of interpolation and convolution is used to reconstruct the image so as to suppress the checkerboard effect and solve the problems of background color distortion,as well as insufficient conversion of the generated image.Experimental results show the FID(Fréchet Inception Distance)and PSNR performance improvements of the proposed algorithm in image style transfer.By using densely connected convolutional networks to deepen the network level so as to enhance the extraction of image features and the using expanded convolution to broaden the receptive field,the problem that the style is not obvious when the improved Cycle GAN model is applied to the artistic image stylet transfer is solved.The performance of the proposed model is demonstrated with the experimental results from data sets such as animation,sketch,van Gogh and Ukiyoe styles.All of the generated images are having a strong target style while keeping the original image content intact,which validate the effectiveness of proposed model in artistic style image transfer and its versatility to multi-style transfer task.