Modality-Oriented Graph Learning Toward Outfit Compatibility Modeling

With the increasing level of living,people’s demand for clothing has changed from basic shelter from the cold to compatible outfits.However,not everyone is sensitive to the appreciation of beauty and good at making compatible outfits from a large number of fashion items.Therefore,outfit compatibility modeling,which aims to automatically evaluate the matching degree of an outfit,has drawn great research attention.In general,each outfit comprises various complementary fashion items,and whether the outfit is compatible or not depends on the matching degree of fashion items.The traditional pair-wise outfit compatibility modeling methods decouple the outfit into several pairs of fashion items,lacking a global capture of the compatibility relation among fashion items.Since there is no clear order among the fashion items in an outfit,the sequence-wise outfit compatibility modeling methods cannot guarantee the reliability of the modeling.According to the number of modalities,existing outfit compatibility modeling methods can be divided into single-modal and multi-modal outfit compatibility modeling methods.The single-modal methods mainly utilize the single modality of fashion items,lacking full utilization of multimodal information of fashion items.To comprehensively model the compatibility of outfits,the multi-modal methods integrate the multi-modal information of fashion items.Despite their remarkable performance,these methods mainly focus on the intra-modal compatibility,i.e.,the compatibility relation between the same modalities of fashion items in an outfit,but overlook the inter-modal compatibility,i.e.,the compatibility relation between different modalities of fashion items in an outfit,thereby probably causing sub-optimal performance.To address the aforementioned challenges,a multi-modal outfit compatibility modeling scheme with modality-oriented graph learning is proposed,dubbed as MOCM-MGL,which regards each outfit as an item graph and utilizes the Graph Convolutional Network to thoroughly explore the intra-modal and inter-modal compatibility relation among fashion items.Extensive experiments conducted on the Polyvore Outfits-ND and Polyvore Outfits-D datasets demonstrate the effectiveness of our proposed method.