The Research Of Sketch-based Image Retrieval

In recent years,the data circulating in the Internet is growing explosively with the rapid development of computer network technology and mobile intelligent devices,especially the multimedia information represented by image and video.Image is one of the most important points of entry for multimedia information retrieval.The large-scale images retrieval and the retrieval for special images attract more and more attention of researchers,and become important research topics in the field of information retrievalThis research mainly solves the problem of image retrieval in multimedia information retrieval.First,feature extraction and representation which are the basic problems in large-scale content based image retrieval(CBIR)are investigated,and then sketch based image retrieval(SBIR),which belongs to CBIR,is studied in detail Also the sketch based image retrieval can be divided into two types:class-level SBIR and fine-grained SBIR(FG-SBIR)The main work and achievements of this paper are shown as follows1.In order to solve the problem of data storage in large-scale image retrieval,a framework for large-scale image retrieval is proposed.Deep feature of the image is extracted by deep convolutional neural network firstly and then it is compressed into binary hash codes by hash method,which greatly reduces the storage stress and speed of retrieval.The proposed framework can effectively carry out large-scale image retrieval and can also achieve good performance in the complex large scale database2.Aiming at the problem that there are many differences between sketch and natural image,this research proposes a domain migration generative adversarial neural network framework named DMGAN,which can learn the domain-conversion information between sketch and natural image using generative adversarial training To some extent,it can translate the sketch to the domain where the natural image is located,making the further retrieval task simpler.And it can be used in many ways to meet the needs for different tasks3.In order to overcome the difficulties in training triplet network in fine-grained SBIR,a fine-grained sketch based image retrieval method based on adversarial training for triplet network is proposed.DMGAN is used to assist the training for triplet network,which makes the training process more smooth and efficient,and finally achieves a good retrieval performance.By comparing the proposal with state-of-the-art methods in recent years,the effectiveness of the proposed method in FG-SBIR has been proved4.To solve the problem of the complexity of edge extraction and information loss in class-level SBIR,an end-to-end framework using cross-domain representation learning(CDRL)and similarity learning network(SLN)is proposed.CDRL combines DMGAN and SLN together for learning similarity information and category information,and processes both sketches and natural images directly,which leads to less training steps and stronger portability.Comparing with other state-of-the-art methods on widely used datasets,it is proved that CDRL has high efficiency and high precision.This research is devoted to many specific tasks of content-based image retrieval,and uses a series of methods to improve the performance of retrieval in several aspects.In the future work,we will consider further optimization of the proposed methods and expand it to more tasks in CBIR.