Content Based Video Retrieval

With the development of Internet and computer technology, a large number of video data is produced everyday. So how to get the interested video clips from the massive video dataset quickly and efficiently has become an urgent problem in the field of information retrieval (IR).In this paper, Content-based Video Retrieval (CBVR) is studied in four levels:low-level visual feature extraction, Semantic Indexing (SIN), Instance Search (INS), and Known-item Search (KIS). The main contents are illustrated as follows:For the problem of low-level visual feature extraction, some existing low-level feature extraction algorithms are analyzed. The Bag-of-Visual-Words algorithm is proposed to quantify the high-dimensional SIFT features. To evaluate the performance of the feature, an experiment is designed on TRECVID IACC training dataset. The result shows that this feature is effective.For the problem of Semantic Indexing (SIN), a novel framework for video semantic concept detection is proposed. A total of eight kinds of low-level visual features about color, shape, texture, and key points are extracted from each video key frame. A support vector machine (SVM) classifier is trained for each feature. The semantic detection result is obtained by decision level weighted fusion of SVM classifiers. In Semantic Indexing task of TRECVID 2010, the evaluation result of our system is higher than the average performance of all the participants.For the problem of Instance Search (INS), we propose a novel framework with three kinds of video retrieval methods which based on face recognition, color information of clothes, and the global image. In TRECVID 2010 Instance Search task evaluation, the performance of our framework ranked the second place, and is much better than the average of all participants’ results.For the problem of automatic Known-item Search (KIS), the textual query is pre-processed and then searched in video metadata and video data respectively. Text-based Retrieval (TBR) method is used in video metadata and voice script data transformed from voice in video by Automatic Speech Recognition (ASR) technology. Content-based Retrieval (CBR) method is used in visual and auditory information of video. Re-ranking the original result list of Text-based Retrieval method by the result of Content-based Retrieval method, the final result of our automatic KIS system is acquired. In Know-item Search evaluation of TRECVID 2010, the result of our framework ranked the second place among all participants’results, which shows our algorithm is effective.