Inpainting-based Virtual Try-on Network

Shopping online has become a lifestyle these days,especially for fashion items.Nearly all customers are concerned about whether the clothes style is suitable for themselves.However,most fashion online stores only provide photographs of these products or the models wearing them,which are not intuitive enough for shoppers to make a buying decision.Therefore,the virtual try-on system,which allows users to try on various clothes in cyberspace,is expected to deliver promising results to solve this problem.Image-based garment transfer,as one of the virtual try-on approaches,aims to swap the desired clothes from a model to arbitrary users.However,existing works cannot provide the capacity for users to try on various fashion articles according to their wishes,i.e.,users can decide which article(e.g.,tops,pants or both)to be swapped.This paper proposed an Inpainting-based Virtual Try-On Network(I-VTON)which allows the user to try on arbitrary clothes from the model image in a selective manner.Firstly,the garment texture was extracted via texture remapping in terms of the Dense Pose results.Secondly,a parsing result was synthesized by Human Parsing Transfer.The user texture was extracted according to the intersection of the synthesized human parsing and the user human parsing.Thirdly,the texture from the garment and the user were combined respectively to form a coarse result.Finally,the missing regions in the coarse result were recovered via a Texture Inpainting Network.Both Human Parsing Transfer and Texture Inpainting Network were trained by semi-supervised learning.The entire design allowed users to choose which clothes they hope to try on via an interactive texture control mechanism.A triplet training strategy and a skin loss were introduced to ensure the naturalness and correctness of the try-on results.Qualitative and quantitative experimental results demonstrated that I-VTON outperforms the state-of-the-art methods on both the garment details and the user identity.It was also confirmed our approach can flexibly transfer the clothes in a selective manner without redundant networks.