Criteria And Algorithms For Structure Analysis And Summarization Of Video Content

In recent years, video content analysis has become a very active research area in the field of video signal processing. The main purpose of video content analysis is to develop a system that can automatically parse, classify and manipulate video data according to its content. Typical applications include large-scale video database and content filtering of home video. The corresponding key techniques are video segmentation, abstraction, indexing and retrieval. In this dissertation, research emphases are placed on some of the aspects mentioned above and several efficient algorithms and reasonable criteria are proposed. Furthermore, the future trends of this research direction are prospected.For video temporal segmentation, a so-called "robustness criterion" is proposed to evaluate shot boundary detection algorithms. With this criterion, two good algorithms are designed: inertia-based cut detection algorithm, and cut detection algorithm with constant false-alarm ratio (CFAR). The latter one translates the CFAR processing technique from radar signal detection into the field of video content analysis. Then, considering the film editing rules, the author points out and researches on a new important research direction, fine structure analysis of video shot transition, for which a soft template-matching algorithm is developed based on the adaptive gradient temporal slice.For video abstraction, the author first develops two efficient key-frame extraction algorithms: the leaky bucket like algorithm and the motion entropy maximization algorithm. Then the author models key frame extraction as a shot reconstruction process. As a result, a new criterion called "shot reconstruction degree (SRD)" is proposed to evaluate the key frames selected by various algorithms. Discussions show that this new criterion is a better and stricter one than the widely used fidelity criterion, and can help to fuse the key frame concepts for both video coding and video analysis. According to this new criterion, a novel recursive algorithm is developed and proved to perform with both good fidelity and SRD. Inthe next part of this dissertation, the author researches on the video skimming technique. A two-layer approach is designed which takes into consideration the film production rules, the relationship between shot entropy and skim ratio as well as the content emphasis distribution within a shot. Subjective evaluations show that this approach leads to a very good dynamic abstraction of the video, although no semantic understanding is needed. At last, the author proposes a global-optimized video signal-processing framework, which aims at solving the video content analysis problem from the basis. In this new framework, all the video signal processing units, including video capturing, video editing, video coding and so on, will consider how to bring convenience to the following video content analysis phase. For this purpose, the two-sensor capturing system is adopted and the formats of both original and compressed video sequence are re-designed. To summarize, this dissertation regulates some sub-areas of video content analysis, develops many effective and efficient algorithms, and enriches the research tools of video content analysis. Also the author tries to deal with the difficulties of video content analysis from a global point of view, which provides the future development of this research direction with a meaningful reference.