Research And Implementation Of Image Stylization Based On Semantic Segmentation

With the improvement of hardware computing power,deep learning has made significant breakthrough,gradually becoming a hot topic in image recognition,image semantic segmentation,image stylization,and many other visual multimedia computing areas.Additionally,as an important research direction in the field of multimedia computing,image stylization is naturally favored by more and more researchers and lots of applications are developed.For example,Prisma,a very popular mobile application in 2016,is the achievement in this field.The theoretical basis of Prisma is the application of deep learning in the field of image stylization.Deep learning as a kind of parameterized learning algorithm,is more flexible and has the advantages beyond traditional visual calculation method.However,as a new algorithm,deep learning in image stylization is still in the prototype stage,and there are a lot of the details of the problem to be solved.In this thesis,the author studies the image stylization algorithm based on VGG-19 and put forward the improved algorithm achieved by the popular deep learning framework to improve the existing algorithm.The thesis mainly studies the convolutional neural network algorithm which is proposed by Gatys et al.in the field of image stylization.The algorithm defines the content-style loss function using feature matrix which is extracted by VGG-19's convolution layer and then the loss function is plugged into the neural network to optimize the generated stylized image by gradient descent iteration.The algorithm has better transformation effects beyond the traditional algorithm of generating stylized image in combination with the source image style and image semantic content,and the algorithm is also more flexible than traditional algorithms.However,during the style transformation,there is a serious problem in the transformation of the foreground and the background style which causes the generated stylized image into a mess due to the lack of prior knowledge that people have inherent in visual perception.Therefore,this thesis proposes an algorithm to optimize the stylized image generation by cascading image semantic segmentation neural network model.Compared with the original algorithm,the algorithm proposed in this thesis has the following improvements and features:1.The problem that the original convolutional neural network algorithm exists is solved by the algorithm proposed in this thesis,which makes the image quality improved significantly.2.The objects in real photos are combined with the art images to generate a relatively novel stylized image by the algorithm proposed in this thesis.3.The iteration times needed to generate the stylized images is reduced by the algorithm proposed in this thesis,and stylized images can be generated better and faster.4.The Input images can be segmented automatically,which solves the problem of traditional semantic segmentation algorithm that human intervention is needed.