Research On Image Style Transfer Based On Semantic Matching And Style Sampling

Image style representation and transfer technology has been widely used in image editing,image creative design and other fields,and is also one of the hot research directions of computer vision and deep learning.Although this direction has achieved many milestones,it is far from mature.There are still two key problems to be solved urgently:1)There are semantic mismatching problems in style transfer,such as transferring the background texture of style image to the foreground of target image,the transfer process does not maintain semantic consistency;2)The style diversity is poor,that is,the transfer algorithm can only deal with a single style or a small number of styles,can't transfer any style.In view of the above problems,this paper studies the development status of image style transfer,and summarizes the relevant technologies.On this basis,the following two works are done:1.To solve the problem of semantic mismatching,a style transfer algorithm based on semantic segmentation is proposed.The algorithm can automatically extract the semantic information of objects in images,and use the semantic information as a constraint to do style transfer.Experiments on WitiArt[1]and CelebA[2]datasets show that,compared with existing semantic style transfer algorithms,the proposed algorithm does not need to manually produce semantic information,but automatically extract it from the network.In addition,while maintaining semantic consistency,the results of style transfer are better than existing algorithms in clarity and detail.2.To solve the problem of poor style diversity,a style transfer algorithm based on VAE is proposed.This algorithm combines the theory of style transfer and VAE.Under the condition of fixed content,it can randomly sample style features and generate images with the same content and random style.Experiments on COCO[3]and WikiArt datasets show that the algorithm can sample styles and synthesize high quality images quickly.In addition,compared with the existing style transfer algorithm,the synthesis speed of the algorithm is 50 times faster than that of the existing algorithm with similar results.To sum up,the contributions of this paper are as follows:1)A image style transfer algorithm based on semantic segmentation is proposed,which can automatically extract semantic information and solve the problem of semantic mismatch in style transfer;2)A image style transfer algorithm based on variational automatic encoder is proposed,which realizes style sampling and solves the problem of poor style diversity.