High-Level Video Semantic Concept Detection Based On Multiple Features

The field of video indexing has been of a rapid growth in recent years, while the problem from semantic gap comes forth when the need for machine to understand semantic concepts like human is more and more urgent. Hence, detection of high-level video semantic concepts has been an active research topic for next generation multimedia search engine. In this paper, we present a generic scheme for detection of video semantic concepts based on multiple visual features, and propose some new fusion algorithms. Our main work are listed below.On the low-level feature extraction, we summarized and analyzed a large number of visual feature extraction methods, including global features such as color, edge, texture, and local descriptors like SIFT and HOG. Especially for local features, Bag-of-Feature method is used to organize the key-point descriptors as histograms, which makes a good presentation and speed up the training phase. Based on this framework, pyramid matching method is used to describe the layout information of key-points, and PLSA is used to model the context relations of visual words and carry out dimension reduction.On fusion algorithms, we investigate the fusion technology in feature level, kernel level and classifier level for video concept detection systems. A linear weighting algorithm based on logistic regression model and a kernel combination algorithm using multiple kernel learning were proposed to combine the advantages of different visual features and make more robust detection for different semantic concepts, including scene, object, event and so on. Based on these fusion algorithms, we present a cascade fusion scheme, which performs better than single algorithm in experiments. Benefit from these algorithms, our video semantic indexing systems got 3rd place in performance evaluation of both high-level feature extraction task of TRECVID 2009 and semantic indexing task of TRECVID 2010.