Large Scale Video Retrieval And Feedback With Multi-level Content Represeentation

With the rapid growth of digital video content production on the web, content-based video retrieval (CBVR) has been receiving increasing attention over the last decade. In the computer vision and machine learning community, many approaches focus on multimedia information indexing and retrieval techniques. Compared with individual images, videos have much richer content and therefore need a more complicated structure to describe, index and retrieve.This paper analyzes the data characteristics of video content and focuses on the four demands of practical applications including large-scale, high dimensions, semantics and fast retrieval. Some challenging tasks are also present such as hash-based appropriate nearest-neighbor search, relevance feedback learning, multi-level presentation of video content, distributed retrieval in massive datasets, video copy detection and semantic learning in content-based video retrieval. The main contributions are illustrated as follows.1. A scalable peer-to-peer indexing scheme for high dimensional nearest neighbor search in massive dataset using locality-sensitive hashing is proposed. LSH indexing structure and DHT network topology are introduced and a nun-uniform Hilbert Curve is present to map LSH bucket labels to Chord node rings. Also a framework of distributed indexing and retrieval is designed to accelerate the retrieval and expand the indexing from centralized settings to distributed scenarios. Experimental results show that in a Chord ring containing5000nodes, the proposed approach reduce the hops of retrieval by40%and the number of working nodes by30%.2. A load balancing and maintenance scheme in structured P2P network for locality-sensitive hashing is proposed. Based on the distributed similarity search system using LSH and load balancing schemes using virtual nodes in DHT, after discussing the particular load balancing problem with mapping the multi-dimensional LSH bucket space to the DHT naming space, a chord-based structure using virtual nodes to perform load balancing algorithms is present. Simulations on the OverSim platform demonstrate the effectiveness of the proposed method by50%load balancing gains with about13%additional communication cost. 3. A computationally efficient algorithm for large scale near-duplicate video detection is proposed. Large scale video copy detection is very desirable for web video processing especially the computational efficiency is essential for practical applications. Based on multi-level video content analysis, local features are extracted from key frames of videos and indexed by an novel adaptive locality sensitive hashing scheme. By learning several parameters, fast retrieval in the new hashing structure is performed without high dimensional distance computations and achieves better real-time retrieving performance compared with state-of-the-art approaches. Then a descriptor filtering method and a two-level matching scheme are performed to generate a relevance score for detection. Experiments demonstrate the efficiency gains of the proposed algorithm.4. An incremental relevance feedback learning method for large scale content-based video retrieval is proposed. By leveraging local feature detectors feature descriptor are extracted from video collections. Then multi-level matching is performed after indexing and retrieval of feature vectors using the state-of-the-art techniques in content representation, similarity measure selection. A novel incremental relevance feedback approach based on canonical correlation analysis (CCA) is introduced to bridge the gap between semantic notions of search relevance and the low-level representation of video content. Experimental results on real world demonstrate the precision and recall gains of the proposed method.