Study On Content-Based Video Analysis And Retrieval

With the rapid development of multimedia and network techniques, efficient and effective image and video retrieval becomes more and more important due to teeming of vast image and video media in many coding formats. Hence, it comes forth a new research field in the information technology, i.e. Content-Based Video Analysis and Retrieval (CBVAR).CBVAR means video retrieval from enormous video database through matching of patterns derived from video analysis and feature extraction to represent its content. In a video, the contents include its structure, low-level visual and aural features, and high-level semantic information, which form the base for further retrieval and visual production. It is still a great challenge to integrate all the information above to establish a general model for content-based video analysis and retrieval. Based on MPEG-7, we have developed such a kind of model in prototype. The main work and contents in this thesis include:1) Based on analysis of the characteristics and difficulties of video and video management, a general video data model, compatible with MPEG-7 standard, is established. In our model, videos are represented in a hierarchical structure with four layers from frame, shot, scene to story unit, and key frames, etc. are introduced to describe the features and contents of layer. The whole structure model makes up a frame for further processing;2) Principles and methods of several typical shot changes are introduced, and different shot detection algorithms are analyzed and evaluated. An improved self-adaptive shot detection algorithm is then proposed, which can identify both saltation and gradual shot changes. Furthermore, a simple and effective method for key-frame extraction from video shot is given. Finally, shot detection from compressed video, based on DC image and motion parameters, is analyzed in detail;3) A spatio-temporal shot similarity rule is proposed to cluster video shots into video scenes, and fuzzy-clustering method for key frame extraction from video scenes is also given. In addition, the concept of scene centriod is introduced to represent video scene by averaging of all key frames inside a group of shots in the given video scene, which makes an audacious attempt for high-level video unit extraction and representation;4) According to the given sample video, shot and scene-based video retrieval methods are discussed respectively, and the evaluation pricinples are also discussed. Moreover, video clipping and merging in MPEG compressed domain is implemented, which will largely exploit network-based applications of CBVAR.