Research On Key Technologies For News Video Semantic Concept Detection And Semantic Retrieval

News video is emerging as one kind of important media type for getting information. For intelligence analyst, news video is also important to assist them in making strategic decisions because of real time reports and publicity. Therefore, study on news video analysis technology is significant for common people and intelligence analyst. But there is a problem that makes the vedio semantic content analysis embarrass. The problem is “the semantic gap” which is between the high-level semantic and the low-level features of the vedio. And the charactors of vedio make browsing and retrieving hard, such as the non-Structured data format, the huge quantities of data, and so on. Therefore, the analysis on the vedios semantic content is significant which can assist people to get more effective semantic information of vedio. This thesis is aimed at solving the key technology of news video concept detection and semantic retrieval, and also supporting the technical assist on structuring an effective system of news vedio semantic retrieval, putting forward an solution of achieving the semantic content of news vedio and striding over “the semantic gap”.Firstly, the architecture of news video semantic retrieval is presented. Then, the key techniques of semantic concept detection based on inter-concept correlation and multi-cue information, multi-level semantic indexing of news video and query expansion are considered. Experiment results validate the feasibility and effect of these techniques. The primary contributions of the thesis are as followed:1. Two kinds of semantic concept detection approaches based on inter-concept correlation are proposed. One is based on implicit context space, correlative concopts selection model is added to semantic concept detection framework based on CBCF(context based conceptual fusion), get the final confidence scores with fusion of visual similarity and detector precision, then a improved boosting algorithm is proposed to limit weight extension and balance FAR(False Rejection Rate) and FRR(False Rejection Rate), which adjusts the weight distribution strategy. Another is based on explicit context space, uses individual detetor scores to explore pairwise concept correlation, and transforms refine process to random walk process, that is decomposed into positive and negative correlations to refine with successively accurate iteration, then obtains robust scores with romoving unreliable correlative concepts.2. Semantic concept detection approaches based on multi-cue information are proposed. Multi-cue information including motion information, temporal information and textual information is analyzed to assist with detecting semantic concepts. With motion information, a concept detetion framework of motion feature fuisoned with background feature is proposed, in which Mo SIFT(Motion Scale-invariant Feature Transform) feature is extracted from video shots, then the PLSA(Probability Latent Semantic Analysis) method is used to extract latent semantic of shots to build latent motion semantic concept detectors. With temporal information, qualitative and quantitative analysis is used to measure several pairwise concept patterns among shots, including temporal consistency, inter-shot context and intra-shot context, then a temporal smoothing framework with temporal context is proposed, and high-order temporal relationship is used to resolve sparse data problem in an extend N-gram language model.3. A hierarchical semantic indexing structure is proposed, including latent concepts, semantic concepts, textual concepts and concepts cluster. And a news story clustering method based on tetrapartite graph partition is proposed, in which a tetrapartite graph of news story is constructed based on analyzing news videos’ high-order heterogeneity, then a multi-objective optimization problem is transformed by decomposing tetrapartite graph into three bipartite graphs that can be solved by semidefinite programming.4. A query expansion method based on multimodal information is proposed, in which preliminary query-to-concepts is solved by similarity between query text and semantic concepts with concepts correlation measured by Word Net, a multimodal mutual framework is proposed to reranking preliminary retrieval results, and then multimodal query is generated to be mapped to hierarchical semantic indexing.5. A prototype system for news video semantic retrieval is designed and implemented. The design thought, the overall architecture and the running process are elaborated hierarchically, which presents a solution to the applications of the frame and relevant methods, are also elaborated.