Research Of Video Semantic Information Extraction

With the development of computer and internet, there is an explosive increase of video data. Contrast to the traditional text data, video is rich, complicated and vivid. However, allow for the Non-structured and multi-meaning content, it is difficult to index the video effectively. It becomes more important that how we organize, store, manage and index videos. The traditional query method can't get video content automatically, so it can't meet the need of information technology. In the early 1990s, people began an new research that draw much attention, which is content-based video retrieval(CBVR)[10,11].The traditional method need to browse and label information manually. It can't generalize the video content objectively, accurately, and roundly. Now the CBVR is based on low feature, divide the videos into shots, choose the key-frames from every shot, and then get the motion feature and visual feature, finally store them in database [3]. System will feed back the most likely results according to the submitted query by people. This method operates on low feature directly, not belong to semantic level. However, people like judge the comparability on semantic level (high feature). The semantic gap between low feature and high semantic feature makes CBVR not applicable for general users. How we fill the gap, and search on semantic level, becomes an challenging research.In this paper, we put our emphasis on semantic extraction. We take advantage of SVM (support vector machine) to extract the semantic feature from videos. With the training data insufficiency problem, we propose an interactive active-learning method, with the less human efforts, we can get better results. However, The existing active-learning usually take single-modality. Concatenating all features as one feature will lead to lots of problems. So we develop an new method: multi-modality active-learning. In each turn, the concept that is expected to get the highest performance gain is selected from each modality. After that, a SVM method is applied to each modality. In this way, the human efforts can be sufficiently explored. The main job in this paper is as follows: first, both the job we have done and the research state of content-based video retrieval are described. Then we study the knowledge about video, like shots segmentation, key-frame abstraction and then propose the method for feature extraction. Third, because of the widely use of SVM, we learn about statistical theory and Support Vector Machine, attempting to apply SVM to video annotation. Based on the former two steps, we propose a multi-modality active-learning method, and validate the proposed method. Also a CBVR framework is designed to help us finish the jobs. Finally, we conclude in section, together with a discussion of future work.