Research On Content-based News Video Mining Methods

In the information age, News video, which is one of the most representative multimedia information source, emerges and plays an important role in politics, economy, culture, diplomacy, and national defense as well as in our daily life. Thus, it becomes necessary to study how to extract valuable knowledge from large news video data sets. Though traditional multimedia processing and content-based information retrieval technology have produced satisfying results in organization, management, browsing and retrieval of multimedia data, they are unable to acquire latent high-layer semantic knowledge. Although the traditional data mining technique can solve the problem of knowledge discovery from structured and semi-structured data sets to some extent, it is difficult to be applied directly to the unstructured multimedia data. Integrating content-based video processing technology and traditional data mining technology, this thesis aims to discover the valuable and comprehensible pattern knowledge from massive multimodal news video data sets.This thesis first proposes architecture of news video mining, and then focuses on the study of the key technologies within the framework such as news video semantic structure mining, semantic topic mining and semantic event mining. The main research work and innovations are as follows:(1) Architecture of news video mining based on content, which includes conception architecture and technique architecture, is proposed. In the conception architecture, the thesis defines conceptions and tasks of news video mining, and presents a hierarchical framework of news video mining. In the technique architecture, the thesis proposes a system structure of news video mining introduces key techniques of news video mining and points out the problems this thesis concentrates on.(2) Some news video semantic structure mining methods are proposed and improved. This thesis proposes a news video shot detection method based on image segmentation and object tracking, and an audio shot detection method based on announcers'voiceprint, and a news video story unit detection method combined audio and video clues, which lays solid foundation for news video high-level mining. The results of the experiment demonstrate that our methods are robust to changes in low-level features, motion of objects and camera as well as differences in the quality of news videos and have high accuracy even when the video quality is pretty poor, and performs very well in terms of precision and recall values.(3) A news video semantic topic mining method based on multi-wing harmoniums models is proposed. Based on a class of bipartite undirected graphical models named harmonium, we propose news video multi-wing harmoniums (NVMWH) model that represents story unit as latent semantic topics derived by jointly modeling the transcript keywords, color histogram and edge histogram features of news video data for news video semantic topic mining. The model extends and improves on earlier models based on two-layer random fields by capturing bidirectional dependencies between hidden topic aspects and observed inputs. The results of experiment show that our model facilitates efficient inference and robust topic mixing, and potentially provides high flexibilities in modeling the latent topic spaces.(4) Some news video semantic event mining methods are proposed and improved. Firstly, this thesis proposes a news video event representation based on multi-semantic classes and calculating methods measuring the similarities between news video events. Secondly, a news video event detection method based on incremental k-means is improved, which overcomes the disadvantages of the choice of initial cluster centers and dependence on the order of processing news corpus, enhances dramatically the news event detection performance. Finally, a mining method of the news event multi-dimensional frequent pattern based on multi-semantic classes and a visualizing method based on"events cube"association graph are proposed. The mining method first establishes multi-dimensional transaction database and then mines the news event multi-dimensional frequent pattern by means of extended Apriority pruning strategy and multi-dimensional index tree. The experimental results demonstrate that the proposed methods are effective and highly efficient and the association graph based on events cube is an intuitive and interesting visualization tool.To sum up, this thesis focuses on content-based news video semantic mining, proposes conception architecture and technology architecture of news video mining, and researches news video mining methods and applications from the aspects of semantic structure, semantic topic and semantic events. The achievements of this thesis will not only promote the development of news video analyzing and data mining, but also have great theoretic and realistic significance in multimedia intelligence analysis.