Study On Image Style Transfer Algorithm Based On Deep Learning

Image style transfer is a significant image processing task that utilizes the existing artistic work to reproduce creation.Given a content image and a style image,style transfer technology can appropriately integrate the style into the content image,generating a new picture with certain characteristics.During recent years,the appearance of neural networks,especially convolutional neural networks,have accelerated the development of style transfer algorithm,making it widely applied to artwork creation,font style transfer,film effects rendering and mobile equipment photograph rendering.Recent style transfer algorithms can be generally classified into two schemas,including image optimization-based and model optimization-based style transfer.Among those image optimization-based methods,they usually regard all the pixel of a picture as the parameters,and make the stylized image fit the features distribution of style image by an iterative way to modify the values of pixels in a picture.Those methods own a favorable flexibility,while they also experienced the problems of content information missing.Among those model optimization-based methods,the elaborately constructed models obtain the feature information of the style by training the models,having the ability to map the content image directly into the stylized image.Those methods extremely increase the efficiency of style transfer,but much time is consumed to train the models and those models are commonly limited to single style.In this paper,we study on the above two categories of style transfer and make improvements on network structure,loss function,methods of feature transfer,making up for the shortcoming of existing algorithms.Our main work includes:1)We analyze the character of image feature representation extracted by various layers of deep convolutional neural networks.The extracted features of different layers are visualized by back propagation algorithms.We determine which layer is used to extract the content and style by studying the extracted features of various layers of the network,which lays a solid foundation for the following sections.2)We propose a cross-training based style transfer algorithm to address the issue of content information losing,object edge warping and color covering in classical neural style transfer.The content image and style image are firstly preprocessed by graying and data strengthening.The content information is preserved as much as possible by executing different total loss functions in turns.3)We propose a real-time style transfer algorithm in the basis of a multi-path feedforward neural network.A style transfer network and a loss network are constructed and the instance normalization is introduced after the convolutional layer to accelerate the convergence of the network.The activation function in the output layer can be adjusted to generate high-quality stylized image.4)We propose a multi-level feature shift based arbitrary style transfer algorithm which contains a highly-trained universal decoder.The feature maps extracted by encoder are directly operated by feature transfer to make the covariance matrix of features of content image match the covariance matrix of features of style image.Two parts are included in feature transfer and they are decorrelation and correlation match operations.We execute the feature shift process by a multi-layer and circulatory manner to further render the content image.