Video Retrieval And Semantic Information Extraction

With the development of network and multimedia technologies,video data is expanding rapidly. So how to effectively retrieve the interested video information from the large-scale dataset has become an urgent issue. Therefore, the Content-Based Video Retrieval has received great attention.In this paper, Content-Based Video Retrieval is studied at three different levels:low-level visual feature extraction,high-level semantic feature extraction and content-based video search,and some novel algorithms and frameworks are put forward.The major tasks in this paper are:In the low-level feature selection and extraction,a large number of visual features are extracted and analyzed in this paper, which can be summed up in four categories:key-point feature, texture feature, edge feature and color feature.First of all, the Bag-of-Visual-Words algorithm is proposed to effectively quantify the high-dimensional key-point feature. Then, the feature fusion strategy between SIFT and SURF is explored.At last, experiments are performed on TRECVID datasets to evaluate performance of different visual features.The experiment results show that the fusion between SIFT and SURF can significantly improve retrieval performance.In the high-level semantic feature extraction, a novel framework for video semantic concept detection is proposed, in which the color, Gabor wavelet, edge histogram and SIFT are used as visual descriptors and a support vector machine is trained for each feature as classifier. After decision-level fusion among classifiers, conceptual test results are acquired.Then various decision-level fusion strategies are put forward in this paper, and are evaluated in self-test experiment, which shows that the mix fusion strategy improves the retrieval performance best by mixing best fusion strategy in each concept.The evaluation results of TRECVID 2008 HLF show that the system's overall detection performance is higher than the average detection performance of all the participants.In the video search,the semantic-based video search framework is proposed, in which the visual example based search approach and the semantic concept based search approach are analyzed.Additionally, the semantic similarity based method and the example correlation based method,are used respectively to establish the mapping relations between concepts and semantic queries, so that the semantic information could be extracted automatically and the video search task is completed.In the TRECVID 2009 automatic video search evaluation, the performance of our framework ranked the first place among all participants,fully verifying the effectiveness of our algorithm.