Sketch-Based Image Retrieval Based On Convolutional Neural Network

Sketches provide a convenient and intuitive way to specify object appearance and structure.As a query modality,they offer a degree of precision and flexibility that is missing in traditional text-based image retrieval-a sketch speaks for a “hundred” words.Closely correlated with the explosion in the availability of touch-screen devices,sketch-based image retrieval(SBIR)has become an increasingly prominent research topic in recent years.Although the advantages of sketch based image retrieval,there are still many difficulties in practical applications.Key challenges for conventional SBIR include but are not limited to: 1.Sketches and images are from inherently heterogeneous domains.2.Sketches are often highly abstract in representation compared with images,different people show a huge gap among their drawing skills and recognition.3.With the need of the people,they want to capture fine-grained variations of objects.4.The convolutional neural network and the feature descriptors are not suit the sketches.In order to solve the existing problems in sketch image retrieval,this paper starts from the features of sketch,and studies the various network structures.A multi-branch fusion sketch convolutional neural network is proposed.Then,this paper propose a deep multi-layer sparse fusion feature descriptor of the sketch.the main research content is as follows:a)This paper investigates advanced sketch based image retrieval methods in the world and describes the related technologies in detail.We analyze the problem of sketch based image retrieval technology,and proposed new approaches.b)In view of the weak adaptability of current convolutional neural networks to sketch based image retrieval,combining the differences between sketches and photoes.We propose a multi-branch fusion sketch convolutional neural networks,which is specifically designed to release the domain gap between sketch and the general image.c)According to the detail analysis of the instance-level image retrieval,and the principle of the attention model.In order to achieve the instance-level image retrieval,we use the attention model to attract the detail information.d)To improve the description power of features for sketches,this paper starts from the network structures,and analysis the features of the different layers.In this paper,we present a deep multi-layer fusion feature descriptor,which improve the accuracy of feature matching and make the feature more robust.Finally,we compare our method with the current state-of-the-art sketch image retrieval approaches.The experiment verified that our method have better results on sketch based image retrieval.