| While the COVID-19 epidemic has not yet been completely resolved,online consumption is booming.Consumption models which are represented by online shopping,sharing platforms,remote office,and online education have sprung up.Fashion,as the main body of daily consumption,consumption based on innovative fashion scenes is ready to emerge through the innovation and integration of online and offline shopping,virtual and reality.The generation of visual content driven by fashion knowledge presents fashion products to consumers in a more multi-dimensional manner.Currently,many research papers have been carried out on the collocation prediction of fashion data.However,clothing image generation based on collocation fashion data is still in its infancy.How to embed collocation rules during the generation process and the controllable generation according to user preferences still need in-depth research and exploration of new generation theories and methods.Specifically,due to the complexity and diversity of fashion images,the generation effect based on fashion data is not satisfactory.The details of generated images are not perfect enough for industrial application.Thus,it is very meaningful to give explicit guidance to the fashion design and generation process based on potential fashion collocation knowledge rules.This dissertation focuses on the in-depth and systematic research on the collocated clothing images generation in the big data environment.The methods we proposed aim to provide theoretical methods and basis to elaborate collocation clothing image generation based on generative adversarial network.The main research contents of this dissertation are summarized as follows:(1)Dressing in clothes based on the matching rules of color,texture,shape,etc.,can have a major impact on perception,including making people appear taller or thinner,as well as exhibiting personal style.This paper investigates clothing match rules based on semantic attributes according to the generative adversarial network(GAN)model.Specifically,we propose an Attribute-GAN to generate clothing-match pairs automatically.The core of Attribute-GAN constitutes training a generator,supervised by an adversarial trained collocation discriminator and attribute discriminator.To implement the AttributedGAN,we built a large-scale outfit dataset by ourselves and annotated clothing attributes manually.Extensive experimental results confirm the effectiveness of our proposed method in comparison to several state-of-the-art methods.(2)In order to tackle the collocation clothing generation between specific domains,we propose a new framework with a multi-discriminator by incorporating different types of conditional information into the discriminator of c GAN for clothing matches.In contrast with most extant frameworks under c GAN,with one generator and one discriminator,the proposed framework investigates the potential of utilizing conditional information delivered by multi-discriminators to guide the generator.Under this framework,we propose a Category-Attribute GAN(CA-GAN)with a generator and three discriminators.Specifically,the framework performs the multi-domain collocation clothing image translation in a uniform framework by co-supervising by categories and attributes.The input of generator consists of clothing images from source domain and categories of target domain.The additional category discriminator aims to predict the category of generated clothing images.In order to evaluate the performance of our proposed models,we built a large-scale fashion collocation dataset.And we also annotated manually a part of clothing images by fine attribute labels.Experimental results demonstrate that,with supervision of the additional attribute discriminator or category discriminator,the quality of the generated clothing images by GANs is consistently improved in comparison to state-of-the-art methods.(3)We explore the collocation clothing images on the multi-modal condition in the generation process.Specifically,we propose a Compatibility Matrix-Regularized Generative Adversarial Network(CMRGAN)for compatible item generation.In particular,we utilize a multi-modal embedding to transform the image and text information of an input clothing item into a latent feature code.Sequentially,compatibility learning among latent features is performed to obtain a compatibility style space.The feature of the input image is then regularized by the style space.Finally,a compatible clothing image is generated by a decoder which is fed by the regularized features.The compatibility learning is based on the triplet sets which consist of clothing image and textual description from source domain,clothing image and textual description from source domain,collocated clothing image and textual description from target domain and non-collocated clothing image and textual description from target domain.To verify the proposed model,we train several evaluated models to measure the compatibility degree of the generated image pairs and a deep attentional multimodal similarity model to evaluate the semantic similarity between generated images and ground truth text descriptions.The results demonstrate the effectiveness of the proposed method on image-to-image translation based on compatibility space.(4)In order to synthesize controllable images of fashion items which are compatible with given clothing images,as well as conditioning on multiple modalities,we propose a multi-modal collocation framework based on generative adversarial network(GAN)for synthesizing compatible clothing images.Given an input clothing item that consists of an image and a text description,our model works on synthesizing a clothing image which is compatible with the input clothing as well as being guided by a given text description from the target domain.Specifically,a generator aims to synthesize realistic and collocated clothing images relying on image-and text-based latent representations learned from the source domain.A text auxiliary representation from target domain is added for supervising the generation results.In addition,a multi-discriminator framework is carried out to distinguish the compatibility between generated clothing images and input clothing images,as well as distinguishing the visual-semantic matching between generated clothing images and the targeted textual information.Extensive quantitative and qualitative results demonstrate that our model outperforms state-of-the-art methods on authenticity,diversity and visual-semantic similarity between image and text. |
