Research On Sketch-Based Image Retrieval Using Deep Learning

With the development of the Internet technology and the explosive growth of image data,demand for retrieving effective information from massive data is increasing.Meanwhile,in recent years,with the development of the touch screen divices like mobile phones and tablets,Sketch-based Image Retrieval(SBIR)has become a more intuitive and effective way for people to retrieval images.Compared with other contentbased image retrieval,sketches are more intuitive and concise,and are more likely to describe key information.Therefore,SBIR has attracted the wide attention of researchers.This thesis analyzes the key issues in SBIR task,and researches on the SBIR technology based on deep learning to improve its performance.In this thesis,we propose a novel edge-guided cross-domain learning model for sketch-based image retrieval.Edge maps extracted from natural images are introduced as the bridge between two domains,thus effectively reduce the domain gap.First,an edge guidance module is proposed to fuse natural images and the corresponding edge maps,which guides the network to generate more discriminative features in the domain alignment process.Second,a shape regression module is utilized to capture the inherent shape similarity between sketches and natural images,and reconstruct the natural images features.Finally,the center loss is introduced to reduce intra-class differences,which ensurs the features mapped to the semantic space is discriminative.By training the proposed network in an end-to-end process,the sketch and natural image domains can be effectively associated,which potentially overcomes the challenge of the common feature learning for two heterogeneous domains.Experimental results on the SBIR dataset demonstrate that the proposed method achieves superior performance compared with the state-of-the-art methods.In this thesis,we also propose a novel semiheterogeneous three-way joint embedding network(Semi3-Net),which integrates three branches(a sketch branch,a natural image branch,and an edgemap branch)to learn more discriminative crossdomain feature representations for the SBIR task.The key insight lies with how we cultivate the mutual and subtle relationships amongst the sketches,natural images,and edgemaps.A semi-heterogeneous feature mapping is designed to extract bottom features from each domain,while a joint semantic embedding is introduced to embed the features from different domains into a common high-level semantic space.To further capture informative features common to both natural images and the corresponding edgemaps,a co-attention model is introduced to conduct common channel-wise feature recalibration between different domains.A hybrid-loss mechanism is designed to align the three branches,where an alignment loss and a sketch-edgemap contrastive loss are presented to encourage the network to learn invariant crossdomain representations.Experimental results on three widely used category-level datasets demonstrate that the proposed method outperforms state-of-theart methods.