Research On Sketch-based Cross-domain Image Retrieval

Sketch-based cross-domain image retrieval(SBIR)uses a sketch as a query to find matching examples in a collection of images,which is an alternative or a complement to the widely used content-based image retrieval(CBIR)and text-based image retrieval(TIBR).Although existing SBIR algorithms have a better improvement than the traditional SBIR algorithm,the problems of feature mapping and cross-domain retrieval of sketches and nature images have not been solved well.Therefore,this paper proposes two sketch-based cross-domain image retrieval algorithms from the perspective of network structure and multi-loss function optimization.In the aspect of network structure optimization,this paper propose a novel hybrid cross-domain joint network for sketch-based image retrieval(SBIR).It consists of a sketch branch and two image branches.The image branch is composed of an image and a sketch approximations network,termed heterogeneous fusion network.In this way,the proposed heterogeneous fusion network is used to find a shared representation,which has line information of sketch while retaining the original information of the image.Then the shared representation is used as image features to reduce cross-domain differences between sketches and images.Besides,triple loss function,data augmentation algorithm and multi-step training is also used to train the proposed hybrid network.Finally,the retrieval results are further optimized by exploring variants of the weight-sharing schemes and different fusion network structures.In the aspect of multi-supervised learning,this paper propose a multi-supervised learning algorithm for sketch-based image retrieval(SBIR).This algorithm employ multi-supervised learning with joint optimization by maximizing intra-domain and inter-domain correlation simultaneously,which can capture the important information of common representation.In view of intra-domain learning,the classification subnetwork and reconstruction learning sub-network are added on top of embedding neural network to predict the semantic labels and preserve semantic consistency of the highlevel semantic features within each domain.In view of inter-domain learning,this paper proposes adding cross-domain co-distributed learning sub-network and triple sorting sub-network after feature extraction network to ensure the same distribution mapping of sketch and color image in the same space and clustering of similar features between domains.Finally,the effectiveness of the algorithm is proved by analyzing the model and comparing it with the state-of-the-art methods.