Font Size: a A A

Research On Content-based Image Search Reranking

Posted on:2011-03-31Degree:DoctorType:Dissertation
Country:ChinaCandidate:X M TianFull Text:PDF
GTID:1118360305466676Subject:Signal and Information Processing
Abstract/Summary:PDF Full Text Request
With the rapid development of recording and storage devices, as well as the significant improvement of transmission and compression techniques, the amount of multimedia data (e.g., image, video and audio) on Internet increases explosively and the video/image-sharing websites become more and more popular. Efficient and effective multimedia search tools are essential for Web surfing. Due to the requirement of high efficiency and the leverage of successful techniques already de-veloped in text search, most of our frequently-employed image search engines, e.g. Bing, Google, Yahoo and Baidu, are implemented by indexing and searching the images'associated textual information, e.g., image file names, URLs, surrounding texts and so on. However, this text-based image search result is not satisfactory because that the textual information is not the essential description of image's rich content. Reranking is then proposed to refine this text-based search result by incorporating images' visual information, user feedback and other information.Although a lot of works have been done on image search reranking, there are still many problems need to be solved, due to the semantic gap between low-level visual features and high-level semantic concepts. In this thesis, we first propose an unsupervised Bayesian reranking method, and then distill six most important problems which should be carefully considered in a practical reranking system, finally proposed semi-supervised active reranking with user feedback and structural learning based supervised topic-aware reranking method. This thesis conducts a deep research on reranking and obtains the following achievements1. By analyzing the intrinsic roles of the textual and visual information in reranking, we propose Bayesian reranking in which the two cues are mod-eled as as prior and likelihood respectively from probabilistic perspective. Bayesian reranking is an general framework and can unify several existing reranking methods. To well model the textual and visual information in Bayesian reranking framework, we also propose to use a local learning regu-larizer to model visual consistency and a pair-wise preference strength rank-ing distance respectively. The experiments conducted on benchmark datasets have demonstrated the effectiveness of the proposed Bayesian reranking method.2. To incorporate reranking technique into practical image search system, there are several issues which will greatly influence the reranking performance, besides the reranking algorithm design. This thesis distills six most impor-tance problems which should be carefully considered in a practical reranking system. the six aspects include algorithm selection, effective visual feature representation, efficient feature extraction, computational cost, the charac-teristics of the text-based reranking, and the utilization of the text-based search results. Their effects to the resulting reranking performance are ana-lyzed based on comprehensive experiments on a dataset collected from three most frequently-used commercial image search engines. We believe that these analysis and insightful findings will provide useful guidelines for the practical application and further research on Web image search reranking.3. unsupervised reranking methods fail to capture the user's search intentions when the query term is ambiguous. Relevance feedback has been proven to be an effective way to solve this problem. However, current work on rerank-ing with user interaction cannot learn the user's intention precisely. This thesis proposes semi-supervised active reranking methods to learn use's in-tention more extensively and completely. This method first obtain the user's labeling information by interacting with users, and then learn the user's in-tention by distinguishing relevant images from irrelevant ones via subspace learning. Furthermore, this thesis proposes a structural information based sample selection strategy to reduce the labeling efforts and a novel local-global discriminative dimension reduction algorithm to localize the user's intention in the visual feature space. Experiments conducted on both syn-thetic datasets and Web image search dataset demonstrate the effectiveness of the proposed active reranking method.4. In image search, the desired result should satisfy both high relevance and high topic diversity. Topic diverse reranking has drawn increasing attentions. However, existing diversified reranking methods suffer from two problems. First, the maximization of diversity and relevance is performed in two-step, which typically will not achieve the joint optimum. Second, visual diversifica-tion, which is used in diversified reranking, usually cannot well approximate the topic diversity due to the semantic gap. In this paper, we propose topic-aware reranking which jointly maximizes the relevance and topic diversity. Through a structured learning framework, the relevance and diversity are modeled by a set of carefully designed features, and then learned from hu-man labeled training samples. The experiments conducted on a web image search dataset demonstrate that the proposed method not only improves the topic coverage compared with existing diversified reranking methods but also improves the relevance compared with relevance-based reranking methods.
Keywords/Search Tags:image search, content-based reranking, Bayesian reranking, visual consistency, ranking distance, active reranking, active sample selection, subspaee learning, topic-aware reranking, relevant reranking, structural learning
PDF Full Text Request
Related items