Research And Practice Of Cross-category Virtual Try-on Based On PF-AFN

The current virtual try-on technology on two-dimensional images mainly studies the same category of changing clothes,such as originally wearing short sleeves,to virtually try on short sleeves,and has not yet realized cross-category virtual try-on such as wearing short sleeves for dresses,wearing suits for short sleeves.In order to eliminate the category limitation of virtual try-on,a cross-category virtual try-on method based on PF-AFN(Parser Free Appearance Flow Network)is proposed in this paper.After inputting a human image and a target clothing image,we first find an "agent" in the dataset,whose body shape and posture are similar to the input human body,and wears the same style of clothing as the target clothing.Then we input the "agent" image and the target clothing image together into the trained PF-AFN model to get the result of the "agent" trying on the target clothing,which provides the user with the effect of trying on a dress as a reference.The main work of this paper includes.(1)In order to study the body type classification and dress style classification,this paper collected and processed 2000 pictures of models and 4008 pictures of different dress styles,and divided the models into Fatty and Skinny according to their body types,and the dresses into Sling,Loose skirt,Loose long skirt,Waist skirt and Waist long skirt according to their styles.And eight classification networks,VGG19,Res Net101,Dense Net121,Dense Net169,Mobile Netv2,Efficient Net-b0,Efficient Net-b4,Efficient Net-b7,were tried for training and prediction of body type and dress style classification.After experimental evaluation,Res Net101 was selected as the classification model in this paper.(2)In this paper,we designed the way to find the human joint angles based on the pose key points,and designed three methods for pose similarity calculation based on the cosine similarity.After experimental evaluation,this paper selects one of them,angle weightless splicing method,as the posture similarity calculation method in this paper.Based on the body type classification,dress style classification and pose similarity algorithm,a human image with the same body type,similar pose and wearing the same dress style as the target dress can be found in the data set,which is called "agent".(3)In order to evaluate the effectiveness of using "agents" to try on the target dresses,this paper collects and produces 1658 sets of dresses for PF-AFN training,and trains the PF-AFN network model.The target dresses and the "agents" are input into the trained PF-AFN to obtain the results of the "agents" trying on the target dresses,which can be used as a reference for the users to try on the dresses.In the quantitative comparison experiment,the FID of the try-on results with "agent" increased by 22.7 compared with that without "agent".In the qualitative comparison experiment,the difference in attributes between different clothing categories can be handled well,and the correct and natural try-on results are generated,which proves the effectiveness of using "agent" for virtual try-on across clothing categories.