A Study On Contour-Based Image Retrieval

With the rapid development of social networks,digital images are spread in a massive way on the web,and are increased by geometric progression.To meet the retrieval needs of people for image contents,content-based image retrieval researches are paid more and more attention and concern.Contour is a very important feature for the human vision system which is able to recognize and distinguish objects in the scene easily and efficiently with it.Thus contour-based image retrieval has both theoretical significance and practical value,and becomes an important research topic in content-based image retrieval field.In this thesis,we focus on developing new models and algorithms for contour-based image retrieval and establish a basic framework for the whole image retrieval process.The research topics include: the invariance of image retrieval and hierarchical salient candidate region estimation,the algorithm of contour feature extraction,database indexing algorithm and re-ranking algorithm.To verify the performance of our models and algorithms,we develop a contour-based real-time image retrieval prototype system,and have done a large number of experiments on it.The main contributions and innovations of this thesis are listed as follows:1.Hierarchical salient candidate region estimation.To solve the scale and translation invariance problem of image retrieval,an estimation algorithm of hierarchical salient candidate region(HSCR)is proposed.With the saliency map,HSCR algorithm is able to capture possible salient regions of objects and selects them as candidate retrieval regions,thus it could provide a few candidate regions automatically for the salient object in an image.This not only makes hierarchical content-based image retrieval possible,but also provides an efficient estimation for the position and size of the salient object in an image.Furthermore we propose an extraction algorithm of image contour saliency map with contour and color saliency of an image.The saliency map obtained from the algorithm can match HSCR algorithm better thus improves the retrieval performance effectively.We do a lot of image retrieval experiments about HSCR algorithm with different saliency detection models and with different contour feature algorithms.The experiment results show that the retrieval precision can all be greatly enhanced.It demonstrates that the proposed HSCR algorithm indeed improves the performance of image retrieval system dramatically.2.Contour feature extraction.A multi-scale feature extraction algorithm based on hierarchical orientation(HOF)is developed.HOF contains the global-to-local information of objects in an image,thus it can give higher similarity to those images which are similar on the whole but slightly different in local parts than images which have highly similar local parts.So HOF is especially suitable for sketch based image retrieval.In addition,the characteristic of HOF can be used for rapid indexing of database to improve the retrieval performance.3.Database indexing and re-ranking.With the global-to-local characteristic of HOF,a topdown database inverted indexing method and a coarse-to-fine re-ranking algorithm are proposed.The top-down inverted indexing method can rapidly remove a large number of images which are not similar to the query image on the whole,so it improves the retrieval efficiency dramatically,meanwhile keeps the retrieval performance.The coarse-to-fine re-ranking algorithm makes a rapid and hierarchical re-ranking for indexing results so that enhances the real-time retrieval performance effectively.4.To improve the performance of existing image retrieval using convolutional neural network(CNN)with sketch queries,we propose an algorithm of image retrieval using binary contour map based convolutional neural network(BCM-CNN).The binary contour map bridges the gap between a sketch and a natural image,and establishes their consistent representation for objects.And with binary contour maps,we can obtain a large number of images like sketches which make it possible to use contour images to train CNN,and ensure the consistency of network layer features of sketches and natural images which include same objects.Thus the proposed algorithm greatly enhances the retrieval precision with sketch queries.5.Development of real-time image retrieval system.A real-time image retrieval system iRP based on a million-level image database has been developed.It supports not only contour based image retrieval iRP(Contour),but also contour and color based image retrieval iRP(Contour&Color)which is expanded from the framework of contour based image retrieval to meet the requirement for color related image retrieval.In addition,the system provides multiple interactive retrieval methods such as sketch,image(such as binary image,gray natural image,color natural image,etc),key word and their combination to users,which make better user experience.Finally,we have collected more than one million images as experimental data from the web,and have done a large number of experiments.With the support of adequate experimental data,systematic performance comparisons are made for different algorithms in each section of contourbased image retrieval iRP(Contour).In addition,experiment comparisons for contour and color based image retrieval iRP(Contour&Color),and image retrieval using BCM-CNN are also made.Experiment results show that,the proposed frameworks and algorithms can achieve satisfactory real-time retrieval results.In contour-based image retrieval,for sketch query,iRP(Contour)obtains higher retrieval precision,and for both sketch and natural image query,the retrieval precision of image retrieval using BCM-CNN is higher than iRP(Contour).