Attention-Based Image Style Transfer

The combination of deep learning and art can greatly improve creative efficiency,which is widely known as image style transfer.Image style transfer which transform the style of a given content image to the style of other art image has received great interest and research in academia and industry.At present,the image style transfer algorithm based on the attention mechanism has become the core of research in the field.This kind of algorithm can ensure that the image can flexibly transform the texture,color and brushstrokes and other stylistic elements,while achieving the consistency of content semantic information and the overall integrity of the image structure.However,at present,these algorithms still have shortcomings in terms of efficiency,image quality and diversity.This paper will improve these aspects.The main work of this paper includes:(1)The existing style attentional-mechanism has high time complexity,so it requires longer running time in the case of high-resolution images,and the overall model efficiency is low.At the same time,the original style-attentional mechanism only calculates at the single pixel level and cannot pay attention to style elements of different sizes,thus ignoring the feature information of different granularities,resulting in distortion and artifacts in the generated results.In order to solve the above problems,this paper proposes a lightweight style-attentional mechanism.By integrating the average pooling and maximum pooling modules in the original style-attentional mechanism,the pooling operation performs key points sampling while constructing a spatial pyramid.Get multi-grained features.This method can reduce the time complexity of the original styleattentional mechanism from O(CN2)to O(CNM),where M<<N.The experimental results show that the method can reduce the time consumption by 34.7% in high-resolution images;in terms of image quality,it can reduce more artifacts and distortions and generate better visual results.(2)There are currently the following defects in the diversity style transfer: the existing algorithm DFP cannot achieve a balance between image quality and diversity at the same time,and the image quality needs to be sacrificed to ensure the diversity effect;The DFP(Deep Feature Perturbation)algorithm is inefficient due to the use of singular value decomposition;The current diversity style transfer algorithm cannot achieve local content control.Based on the attention-based style transfer algorithms,this paper injects noise into the similarity matrix of the style-attentional mechanism.Our method generates diversity results in two ways: by injecting Gaussian noise or using Dropout.On the above basis,local diversity control is achieved by introducing an additional semantic segmentation network along with style transfer model.Experiments show that the method in this paper has improved in terms of efficiency,image quality and diversity.What’s more,our algorithm is more flexible and controllable.