Compatibility Prediction Of Fashion Suit Based On Heterogeneous Graph Neural Network

With the rapid development of the fashion market,consumers’ demand for fashion clothing recommendation is getting higher and higher,and clothing compatibility,as an important factor when consumers buy clothing,has important research significance in the field of fashion clothing recommendation.Although there have been many researches on clothing compatibility,and the current best-performing models have been able to model the relationship between clothing items in the same suit with a graph network,there are still some problems in these researches: First,due to the construction of The principle of random replacement is adopted in the negative sample,which leads to the combination of the same clothing items that are very easy to identify in the negative sample,which makes the recognition effect of the model appear falsely high;secondly,because the difference between different types of clothing items is not considered.The complex relationship makes it difficult to model the compatibility of fashion suits in a more detailed manner;finally,the existing models have imbalanced training of different samples during training.In response to the first problem,this paper proposes a new negative sample sampling strategy,which solves the problem of artificially high effect of the existing model by using the principle of similar replacement to reconstruct the negative sample set;for the second problem,this paper proposes The clothing compatibility prediction model based on the heterogeneous graph neural network,combined with the graph global pooling technology,realizes a more fine-grained modeling of the complex relationship between the items in the suit.There are more even training opportunities.To verify the design ideas proposed in this paper,we implement a HG-FCP model and compare it with some existing representative works based on the Polyvore dataset.The experimental results show that the performance of the HG-FCP model proposed in this paper has obvious advantages compared with other methods,thus illustrating the effectiveness and advancement of the HG-FCP model.In addition to this,to evaluate the robustness of the clothing compatibility prediction model,which looks like a black box to us,in realworld environments,we use natural or semantically meaningful natural Adversarial examples to evaluate the robustness of the model.To this end,we construct a set of natural adversarial examples using different natural adversarial examples generation methods.Experiments show that the prediction effect of each clothing compatibility prediction model in the natural confrontation sample set has dropped significantly.In order to explore the specific reasons for the decline of the model effect,we used the graph neural network interpretability analysis method to analyze the interpretability of our model,and we found that the clothing items that most influenced the model’s decision were often the items that were replaced using the same-like replacement principle in the negative samples,which demonstrated the effectiveness of the natural adversarial sample set we constructed.Finally,we calculated the proportion of the clothing item category corresponding to the "key node" that most affects the model compatibility prediction under the graph network interpretability analysis method in all categories of clothing.According to this result,we optimized the hard negative sampling strategy.Experiments show that retraining the model with the optimized hard negative sampling strategy achieves better results on the test set.Finally,in order to improve the robustness of the model,we add the constructed natural adversarial sample set to the training set for training.The experiments show that the robustness of the trained model has been significantly improved.