Research On Apparel Retrieval Based On Semantic Similarity

The apparel industry has flourished in the era of e-commerce and is now revitalized in live e-commerce.Among the massive apparel image and video data,it has important application value to accurately retrieve goods that meet users’ search criteria.With the continuous development and application of deep learning in the field of computer vision,existing research using content-based retrieval techniques can directly analyze the content of apparel images and videos in a semantic way,however,the semantic information contained therein is often difficult to be fully explored,and the existence of a semantic gap makes the low-level visual features obtained by the computer inconsistent with the high-level semantic features required for retrieval,leading to discrepancies between the actual retrieval results and the users’ search criteria.Therefore,it is important to study a method that can effectively mine semantic information in apparel images and video contents to improve the performance of apparel retrieval.This research topic investigates the semantic similarity-based clothing retrieval algorithm,which relies on the intelligent video operation technology,a cooperative research project of ZTE Corporation.This thesis mines the semantic similarity relationship between clothing by introducing semantic similarity information based on clothing attributes,and conducts research in unimodal retrieval scenarios represented by clothing images and multimodal retrieval scenarios represented by live streaming with goods,respectively,with the following main research contents and innovative results:1.To address the two problems of insufficient semantic information mining in the garment image retrieval problem and incoherent semantics of the learned embedding space due to the fixed margin in original triplet loss,this paper proposes a semantic similarity-based garment image retrieval method.First,the garment attribute vectors are constructed based on the attribute labels,and the inner product between the attribute vectors is approximated to represent the semantic similarity between garments.Then,in order to mine the semantic information in the image content,the semantic similarity is associated with the margin in the triplet loss,so that the size of the margin can change dynamically according to the change of the sampled triplets,which can effectively learn the semantic similarity relationship between garments and obtain a semantically coherent embedding space.In DeepFashion public dataset,the adaptive triplet loss with dynamic margin proposed in this paper improves the accuracy by more than 7%compared with the original triplet loss with fixed margin.2.To address the two problems of insufficient semantic information mining and difficulty in classifying traditional multi-label classification tasks in the multimodal clothing retrieval problem in the live-streaming with goods,this paper proposes a multimodal clothing retrieval method based on semantic similarity.First,by extracting the attribute keywords in image text captions and live video speech,the semantic similarity between garments is calculated according to the method in Research Content I,and the adaptive triplet loss with dynamic margin is used.Then,by designing a multi-task learning-based clothing retrieval and multi-label classification model,the performance of clothing retrieval can be improved by using the correlation between different tasks.In order to reduce the difficulty of the multi-label classification task,the TF-IDF algorithm is used to obtain the word frequency statistical features of the attribute labels and quantify the importance and priority of each attribute label accordingly,which can reduce the difficulty of the multi-label classification task while preserving the key semantic information,and finally effectively improve the accuracy of the multi-task learning-based apparel retrieval task.In the experimental and result analysis sessions,the effectiveness of the proposed algorithm is verified on the Watch And Buy dataset.