Clothing Image Retrieval And Collocation Based On Deep Learning

With the rapid development of e-commerce platform,people are more and more used to buy clothes online.People hope that the e-commerce platform can realize the function of clothing image retrieval,and can provide appropriate clothing matching suggestions when they purchase clothing.Users have such needs and there are a large number of clothing pictures on the Internet.In this context,all major e-commerce companies hope to provide better services to users in the retrieval and matching of clothing to increase their competitiveness.Clothing image retrieval is based on a better understanding of clothing image,and clothing matching is based on a better application of compatibility between clothing.In this thesis,both clothing retrieval and matching are explored by using deep learning.The main work of this thesis is as follows:The capsule network(CapsNet)is modified as the clothing retrieval model in this thesis.The original capsule network has a strong ability of multi angle understanding of objects,but the feature extraction layer has only one layer of convolution layer,which can not be applied to complex real data sets.In this thesis,the residual module with Se-block is used as the feature extraction module of capsule network,which enhances its feature extraction ability and makes it suitable for the field of clothing pictures.Unlike most clothing retrieval models,which use triplet loss to optimize,this thesis uses the margin sample mining loss(MSML)as a loss function to train the model.Experiments on deepfashion dataset show that the model in this thesis not only has fewer parameters,but also improves the recall rate compared with other clothing retrieval model based on convolution network.The graph autoencoder(GAE)model is used in clothing matching.If clothing is regarded as a node in the graph,two matched clothing means that there is an edge between two clothing nodes in the graph,so the graph contains all the information of all matched clothing.The graph convolutional network(GCN)is used as the encoder to learn the collocation relationship between clothing.Because people value different attributes when they choose different types of clothes,a type-aware decoder is proposed under this common sense.Experiments on the Polyvore clothing matching dataset show that in the expanded FITB(Fill In the Blank)task,the model in this thesis achieves the optimal performance in case of different number of options.When the number of options is 4,the prediction accuracy can reach 0.93.In the task of predicting whether the collocation is reasonable or not,AUC value can reach 0.99,which is also the best compared with other modelsFinally,the thesis points out the future improvement direction of the clothing retrieval model and the clothing matching model.For the clothing retrieval model in this thesis,we can improve the model by using more advanced metric learning methods and implementing attention mechanism in the model.For the clothing matching model in this thesis,it can be optimized from the perspective of using more attribute information of clothing and realizing user personalization.