Large Scale Visual Search And Search-based Image Annotation And Reconstruction

With the rapid development of storage and compression techniques as well as the popular of digital cameras and social networks, the number of multimedia data has explored in the last decades. The emergence of large scale databases has provided new challenges and oppertunites to many research problems in multimedia domain, especially image-related applications. How to manipulate these large scale web data and leverage them to advance the research in image applications have become a hot research toic in computer vision and multimedia communities.To explore the great potential of large scale media data publically available on the Web, we begin with the large scale visual search problem and investigate how to establish effective and efficient matches based on local features. To improve the performance of current bag-of-visual words based approaches, we study how to conduct better visual quantization and image representation. Besides the visual search problem, we also investigate how to apply content based visual search techniques to other applications, such as image annotation and reconstruction based image sharing. The main contributions of this dissertation can be summarized as follows:(1) We propose a novel visual quantization method based on weighted K-means tree and co-occurrence information between visual features. Firstly, we distinguish the discriminative power of different local features and construct a hierarchical K-means tree to make sure that the visual features with high discriminative power could be quantized correctly. Secondly, we leverage the co-occurrence information between visual features to speed up and optimize the whole quantization process by prediction and verfication. Experimental results based on the same visual vocabulary have shown that the effectivenss of the proposed approach. Compared with other visual quantization methods, it improves the image retrieval’s performance.(2) We propose a visual group based image representation for large scale image retrieval. Fisrtly, to improve the local feature’s discriminative power, we propose to group the geometrically related features based on the inclusion relationship between local features at different scales. Secondlly, the geometric constrain inside each visual group is introduced into the group matching for efficient geometric verification. Experimental evaluation on large scale image databases shows that our visual group based visual search approach outperforms the bag-of-visual-word approach and the state-of-the-art visual phrase based schemes.(3) We propose an efficient mixture modeling approach for tag mining in search-based image annotation, which formulate the tag mining problem as to rank salient phrases. Firstly, we extract some salient phrases as candidates by evaluating certain statistic properties. Secondly, we propse a mixture modeling approach based on learnt topic space from a human-edited web dictionary. Thirdly, we leverage the mixture model to rank the candidate phrases and output the top-ranked ones as final image annotations. Experimental results conducted on2billion web images demonstrate the effectiveness and efficiency of our proposed approach.(4) We propose a novel mobile-cloud scheme to enable instant sharing of high-resolution mobile images. Firstly, the high-resolution images are described by their thumbnails and local features, which are further jointly compressed to make sure a low latency. Secondly, high-resolution images are reproduced from a large scale image database by retrieving partial duplicate images by local features and stiching corresponding image patches together under the guidance of the thumbnails. The user study of image sharing based on1million image database shows that the proposed approach outperforms current methods on visual quality.