Content Based Video Retrieval

The wide availability of digital sensors, the high bandwidth Internet, and the falling price of storage devices have resulted in the increasing growth of unstructured digital media content. Therefore, large scale information processing technologies is an important matter of great urgency. Since the last decade, the problem of content based video retrieval has been actively researched by many communities. This paper investigates multiple aspects of content based video retrieval, including video segmentation, automatic retrieval and interactive retrieval. All the works are evaluated on the common TRECVID benchmarking platform.We focus on shot segmentation for completing video segmentation. We present a shot boundary detection framework with collaboration of multiple detectors. A motion vector and finite state automata based algorithm for robust detection of gradual transition is also proposed. Evaluation results reveal the effectiveness of our framework.Based on the work of shot boundary detection, we proceed with content based automatic video retrieval. We fulfill the basic retrieval model of each modality, including text, image and concept. We also propose a pseudo feedback based weighting scheme for multimodal fusion. Detailed experiments and analyses demonstrate the effectiveness and robustness of the fusion scheme.Furthermore, we do some primary work on interactive video retrieval. We develop an interactive video search system with some basic interfaces and relevance feedback algorithms. The interactive search experiments are done on different automatic search results. The experimental results show that the temporal aggregating is meaningful for video retrieval and the improvement of automatic retrieval results is also helpful. Relevance feedback is useful when using text retrieval as the first run, but is trivial when using multimodal retrieval as the first run. Meanwhile, the experiments of real users reveal the great impact of user differences on interactive video retrieval.Finally, this paper concludes our works and presents our perspective on the future direction of content based video retrieval.