Research On Integrating Multimodal Information To Automatically Parsing News Video On The Compression Domain

With digital information increases explosively, the unstructured multimedia information brings much inconvenience for information organization, browse and retrieval. The motivation of video parsing techniques is to make the video media more structured. TV news, as an important program type, is a key source for each family to timely know what happened in the world. Intelligent interaction with video content needs the support from the techniques of parsing video. The dissertation addresses the fast and effective parsing techniques, which integrate multimodal information to automatically parse news video on the compression domain. The research focuses on many aspects, including constructing index of MPEG-1,2 streams, shot segmentation, caption detection, anchor shot detection, as well as integrating audio, video and text cues to automatically extract news items and other semantic units. Some impressive contributions are list as follows.1. Propose an effective index model for MPEG-1, 2 streams. Based on the model, the algorithm of constructing index files is designed and implemented, as well as the algorithm to randomly access any frame in a stream. The model can not only fast locate I frames, but also P or B frames easily. The time cost of locating a frame can be approximated a constant, irrelevant with frame positions and bit rate.2. Propose a fast and effective shot segmentation algorithm on the compression domain. Compared with other conventional algorithms, the algorithm does not compare the consecutive frames. The interval of frames compared is adaptively chosen, among three different granularities, i.e., GOP levels, sub_GOP levels, and frame levels. The algorithm uses different raw information in different types of frames, such as DC in I frames, macroblock types in P,B frames, which decreases the computation cost greatly.3. Propose a fast caption detection algorithm on the compression domain.