Instance-aware Image Style Transfer Based On Deep Learning

With the improvement of material standard of living,people have more spiritual needs.The pursuit of artistic beauty promotes the development of image-stylization application.Image style transfer is to use computer technologies to stylize the content,shape and textures of an image,and make it has artistic visual effects.This application is convenient for humans to stylize their photos into famous paintings,and it will play an important role in advertising design,game scene rendering,animation film production and many other fields.While deep learning has powerful learning ability and data processing capability.Its rapid development has accelerated major breakthroughs in the fields of natural language processing,image generation,and semantic segmentation.Therefore,research of image stylization based on deep learning has achieved good results.By studying the existing deep neural network model,it is found that there are still some problems to be solved in the aspects of model training,stylization level and interactivity in the current image stylization algorithm.In order to solve these problems,this paper designs a new image style transfer network,which not only can generate effective stylized pictures,but also can render different target objects in the original image.The work of this paper mainly includes the following aspects:1)In order to realize the instance perception,this paper first studies the basic principles and techniques of Convolutional Neural Networks,then summarizes two semantic segmentation networks: FCN and SegNet,that can segment objects which belong to same class from the background.To segment different individuals in an image,this paper investigates the instance-aware semantic segmentation network: Mask RCNN.As this network can effectively realize the segmentation of instances,it will be a necessary tool in the final instance-aware stylization model.2)In order to achieve fast and fine image style conversion,this paper firstly compares two types of style transfer networks: descriptive style transfer network and generative style transfer network.This paper designs a Style-Content scoring model based on VGG-16 network,namely S-C scoring model.This model is a multi-classifier network.In the last layer of VGG-16,a softmax is used to classify different styles,and the output is the probability of each class,and we call this probability the deception rate.We calculate the style deception rate and content deception rate of the generated image.The higher the two deception rates,the stronger the stylization of the generated image and the more structural information of the original content image retained,so that an overall and objective evaluation can be performed for the quality of the image.3)We propose a new image style transfer network based on conditional GAN.This model takes the target style image features as a condition to the generation network and discriminant network,and alternately trains through the semi-supervised strategy to generate stylized images.A new style encoder is added to the style transfer model to achieve multi-scale,adaptive style feature selection.In addition,this paper also designs a more effective loss function,so that the network can not only capture the characteristics of the target style image reasonably,but also retain the effective information of the original content features.4)The instance-aware semantic segmentation network is used in the whole style transformation model to generate mask,then based on MRF,stylized image can smoothly blend with the original background.Finally,the style transformation of the target object in the image is realized.By using PyQt module in python to implement the final user interface platform,users can easily load input images and pretrained models with sample buttons,rather than through cumbersome instructions of Linux,so the platform makes operations more Convenient and user-friendly.