Research On Virtual Try-on Method Based On Deep Learning

The virtual try-on technology adopts computer technology to predict and visually display the try-on effect of users,which can solve many problems in the current network era and even the current booming metaverse.Virtual try-on technology based on deep learning has greatly improved the try-on effect due to the introduction of cutting-edge AI technology.Despite the fruitful research results of predecessors,there are still some shortcomings,such as errors in the exchange of clothes with different collars,inaccurate deformation near the collars,loose coordination between sub models,and cumbersome training.In order to solve these problems,in this paper we propose a new virtual try-on model.The new virtual try-on model is based on the deep learning technology,which simulates the try-on steps of human beings in the real world.The research is carried out from three parts:(1)extracting human body information,(2)clothing transformation and(3)synthesizing try-on images,and a new algorithm and a neural network framework are proposed:(1)Research on human body representation model(extracting human body information).The human body representation model extracts the information of the human body and removes the information of the clothing,including three parts: the human head image,the human posture and the human shape.It simulates the steps of taking off the current clothing when conducting try-on.In order to solve the problem that the human shape has a vacancy in the neck,a new human shape extraction algorithm that has no vacancy in the neck is proposed.(2)Research on garment transformation modelThe clothing transformation model preliminarily transforms the clothing to be wear,so that it is roughly aligned with the human body and conform to the human posture.Aiming at the problem of insufficient garment transformation ability of garment deformation model,a garment transformation model based on deep learning is proposed,which has the ability of deformation and shearing.(3)Research on try-on modelThe try-on model generates a human body and a synthetic mask used to deal with the occlusion relationship between the human body and clothing,and synthesizes the final virtual try-on image.To solve the problem of insufficient cooperation between the sub models of the model and the tedious training,a training scheme is proposed for the joint training of the clothing transformation model and the try-on model.In order to strengthen the relationship between the sub models,they share the feature extraction part.The experiment is carried out on the VITON dataset.By comparing with the virtual try-on effect of other models,the model in this paper can reduce the number of model parameters,better solve the try-on problem between different types of collars,improve the deformation effect of clothing near the collar,strengthen the relationship between sub models of the model,and simplify the training process.Compared with CP-VTON model,the parameters of this model decreased by 15.1%,FID effect increased by 34.5%,SSIM increased by 3.2%,and PSNR increased by 6.3%.