Automatic Texture Extraction And Synthesis Of Multiple Exemplars Based On Deep Learning

Texture is one of the most important representation methods in digital image and it is heavily used for modeling the appearance of virtual objects.With the development of science and technology,the virtual reality industry and animation industry are increasing.Behind the rise of these industries,they need to rely on complex manual work.That involves a large number of scene image texture processing work,which consumes human resources,costs a lot of money,and delay the production cycle.With the development of computer technology,it is more and more important to apply computer to process texture image.Traditional texture synthesis techniques usually only emphasized on creating an optimal target textures,without paying enough attentions in the obtaining ideal input texture exemplars.Currently,obtaining texture exemplars from natural images is still a labor intensive task for the artists that usually require carefully photography and heavily post-processing.In this paper,we present an automatic texture extraction and synthesis of multiple exemplars based on deep learning.Our previous research is automatic texture extraction of multiple exemplars based on textureness.To improve the efficiency of the dominative texture identification,we first perform a Poisson disk sampling to randomly and uniformly crop patches from a natural image.To improve the accuracy of global textureness recognition,we use a GIST descriptor to roughly distinguish texture patches from non-texture patches based on SVM prediction(trained from the UIUC database and 15-scene dataset).To identify the real texture exemplars consisting solely of the dominative texture,we further measure the local textureness of a patch by extracting and matching the local structure(BGP)and dominative color features(color histogram)between a patch and its sub-regions.Finally,we can obtain the optimal texture exemplars based on both global and local textureness measures via scoring and ranking on each extracted patches.Our method is evaluated on a variety of images with different kinds of textures.Convincing visual comparison with an artist manual selection and statistics results demonstrated its effectiveness.After obtaining many texture exemplars,it is very complicated to find suitable texture exemplars in texture library which has the thousands of texture images.Using the automatic texture synthesis of multiple exemplars based on deep residual network can solve the problem more effectively.First,we extracted the theme color of the input image,and chose the texture exemplars who the color is similar with the color of region.In order to ensure the result of texture synthesis in each region,we measure the textureness,homogeneity and repetitiveness of texture exemplars,compute the synthesizability of texture exemplars.That can further chose the texture exemplars with great synthesizability in each region.Considering the aesthetic compatibility between the different of texture exemplars,we designed a texture exemplars pair dataset according to the texture characteristics and compatibility.We predict the top 3 groups of texture exemplars pairs by the model who train with the dataset used the deep residual network to.Finally,it is synthesized into the input scene image to form three sets of scene diagram schemes with different styles.We designed and implemented an interactive system for automatic texture synthesis of multiple exemplars,and compared the accuracy of deep residual network.