Research On Virtual Try-on Algorithm Based On Deep Learning

The 2D image-based virtual try-on task aims to transfer a given in-shop clothing image to the clothing area corresponding to a reference body image,and the potential applications and commercial potential contained therein have led to extensive research and interest from researchers in recent years.The virtual try-on task starts by deforming the in-shop clothing to align it with the body posture,thus obtaining a deformed clothing image,then generating the arms and neck of the corresponding clothing according to the shape,and finally fusing the aligned image of the in-shop clothing,the generated arms and neck parts,and invariant areas(head,legs,etc.)to produce the desired virtual try-on result.However,current virtual try-on algorithms do not align the clothing well with the body and do not reproduce the original body features well because they ignore arm details too much.These problems result in the current virtual try-on task remaining a significant challenge for researchers.To solve the above problems,this paper proposes two different virtual try-on algorithms,a virtual try-on network based on semantic adaptation and distributed componentization,and a virtual try-on network based on semantic constraints and flow alignment.They are summarized as follows: 1)a semantic prediction module,which predicts the semantic distribution of try-on results through an improved semantic segmentation network,and the resulting target semantic map can be used to guide the alignment of clothing,the generation of arms and the fusion of try-on results,2)a clothing alignment module,which deforms in-shop clothing to align with the body through an improved geometric matching network or the garment appearance flow network,and 3)the try-on fusion module to fuse the aligned in-shop clothing with the invariant areas based on the generated target semantic map by means of improved convolutional neural networks(CNNs),followed by filling in the missing parts of the body.The specific implementation details are as follows: 1)using the human pose points and the in-shop clothing as the guiding conditions for the target semantic map generation,2)using the clothing semantic layer in the predicted target semantic map as the shape condition for the clothing warping,as well as using the arm and neck semantic layers to generate the corresponding parts,and 3)using the target semantic map as try-on layout information,converting the layers to real textures via the generator and seamlessly blending the aligned clothing to the locations of the clothing semantic layers in the reference image to obtain the final try-on results.In summary,in the first framework,we split the human body into three components: arms,neck,and clothing,and design a semantic map-based image adjustment network to optimize and generate each of them.In the second framework,we optimize the accuracy of the target semantic map by means of an improved semantic generation network,and optimize the garment alignment effect by introducing appearance flow.Experiments on a public benchmark dataset(VITON)demonstrate that the results generated by the proposed method in this paper are rich,natural and realistic in detail and superior in terms of metric results compared to state-of-the-art methods in both qualitative and quantitative terms.