The Study Of Context-Aware Texture Synthesis And Its Applications

Texture refers to a class of imagery that can be categorized as a portion of an infinite pattern consisting of stochastically repeating elements.Based on the inherent repeatability,example-based texture synthesis aims to arbitrarily generate a large texture from the input exemplar with similar but not identical visual appearance.It originated from the texture mapping problem,which is a core technology of computer graphics.It is also a basis of many image processing problems and computer version problems like image synthesis,image inpainting,image melding and so on.Example-based texture synthesis techniques can be categorized into two classes:parametric techniques and nonparametric techniques.Among them,nonparametric techniques are widely used for its efficiency,simple computation and robust adaptation.However,most nonparametric synthesis techniques have the following issues:(l)For textures with complex textures,nonparametric synthesis techniques may fail to capture the structures in a long-range,due to the limitation of the size of its patch.(2)The synthesized result is very sensitive to the patch size.A small patch may be not large enough to maintain a consistent structure.A large patch may increase the danger of a large direct copy of the exemplar texture.(3)Most nonparametric techniques use a nearest neighbor search to find the best match patch for texture synthesis.This greedy strategy may result in a heavily repetition of some patches in the synthesized texture.As a result,the global statistics of the synthesized texture would be different from that of the exemplar texture.For the above problems,this research tries to solve them by using the context information of each patch.On the one hand,the structure feature of the exemplar texture could be extracted from the context to enhance the ability of maintaining the structure.On the other hand,the context information could decrease the reliability of the synthesized texture on the patch size.Specifically,this research mainly solves following 3 problems.(1)For textures with complex structures,this paper proposes a non-local based optimization model in the feature space of a convolution neural network.The non-local operator is used to capture the long-range structure to enhance the ability of maintaining the structure.In addition,a new statistical constraint is proposed to constrain the above non-local texture optimization model to ensure the consistency of the statistical distribution between the synthesized texture and the exemplar texture.(2)In the image pixel domain,a general context-aware texture optimization model is proposed,and two kinds of context descriptors are provided as the description of the context information of the image.On the basis of the context information of image patches,the proposed algorithm could synthesize a new texture with small patches.In addition,a patch match algorithm basing on the optimal transport theory is used for the optimization of the proposed model.This method ensures that the distribution of patches in the synthesized texture is consistent with that of the exemplar texture.However,the computational complexity of the optimal transport theory is very high.An efficient patch match algorithm based on the grouping is proposed.It makes use of the characteristics of the optimal transport to maintain the global distribution consistency,and also gives consideration to the computational efficiency.(3)The proposed context-aware texture synthesis technique is applied to the problem of image style transfer.For the image style transfer problem,a small patch usually results in a better result.The proposed context-aware texture synthesis technique is especially suitable for this kind of problem.Inspired by the image restoration model,this thesis proposes a style transfer model,which is based on the texture synthesis technique,and uses the alternating direction multiplication algorithn to optimize this model.Because of the application of optimal transport theory,the consistent patch distribution between the synthesized image and the style image is guaranteed,which leaves out the additional color transfer and results in a better result.