Cross-domain Clothing Retrieval System Based On Deep Convolutional Neural Networks

Online shopping develops rapidly in recent years,which has been an indispensible part of our life.Clothing is one of the most important categories in online shopping.Traditionally,clothing is retrieved by matching keywords input by users to item names in online shops.The keyword-based retrieval methods are quite coarse and it cannot satisfy the need of finding the exact matching clothing,which calls for the study of content-based clothing retrieval.Content-based clothing retrieval enables users to retrieve clothing by photos,which makes it convenient to shop online.The core problem we want to solve in this paper is,given a clothing photo taken by a user,how to find clothing of similar style,or even the exact clothing,which is modeled as a crossdomain clothing retrieval problem in this paper.Usually,photos taken by users are not controlled with significantly varying poses,picturing angles,and lighting conditions,while clothing pictures in online shops are photographed by professionals with clear background,sufficient light,and controlled poses.It is challenging for cross-domain retrieval to minimize the discrepancy between two domains and improve retrieval accuracy.Specifically,in this paper,we focus on how to improve the accuracy and efficiency for content-based clothing retrieval.To improve the accuracy of clothing retrieval,we first introduce a metric learning model based on CNNs(convolutional neural networks),namely Triplet model.Then we introduce two existing strategies in the metric learning model based on CNNs,namely the parameter sharing strategy and the parameter non-sharing strategy.Considering that the parameter sharing strategy can not reflect the difference between the two domains,while the parameter non-sharing strategy double the parameters number of model which reduces the generality of model,we propose a parameter partial-sharing strategy based on the fact that early layers of CNNs learn generic features,such as edges and colors,and thus can be shared across domains while later layers learn semantic features specific to each domain.Experimental results on a real clothing retrieval dataset have shown that parameter partial-sharing strategy performs best or comparable to the best.We introduce the whole process of setting up a prototype clothing retrieval system.To improve the efficiency of clothing retrieval,the system combines deep hash and parameter partial-sharing CNN architecture for cross-domain clothing retrieval,which learns binary hash features and thus greatly reduces the storage of image features and speed up the retrieval.The system prototype can be accessed at http://202.120.39.165:9999/retrieval.The key point of the deep hash is the Triplet network architecture and the objective function.The objective function includes three losses: Triplet loss,hash error loss and information entropy loss.Optimizing the Triplet loss enables the feature network to learn features that are suitable for retrieval;optimizing the hash error loss makes the output feature vector of the deep hash retrieval model as close to 0 or 1 as possible;maximizing the information entropy loss is to ensure that the hash code has the highest entropy to carry as much information as possible.