Research On Content-based Multimedia Visual Information Retrieval

With the great advances in electronic and multimedia techniques, multimedia visual content information which acts a vivid and interesting knowledge representa-tion modality has more and more influence in recent years. Meanwhile, the develop-ment of Internet and large-scale data storage techniques accelerate the storage and propagation of multimedia visual content information furthermore. How to organize, represent, and retrieval these gigantic volume of visual content information has been a focus problem in modern information retrieval community.Concentrating on the overall framework of content-based multimedia visual con-tent retrieval, in this thesis, we research into four aspects of visual content analysis: visual content multi-label concept detection, visual content specific concept detection, visual content label ranking based on semantic relevance and interactive color layout retrieval. The main contributions are illustrated as follows:1. For the problem of visual content multi-label concept detection, we propose a sparse graph based transductive semi-supervised learning method. Conventional methods assume that the concepts happen independently, hence neglect the cor-relation among multiple concepts. We exploit the sparse signal representation theory to mine the visual similarity among instanes and the distribution correla-tion amone concepts. Then, the concept correlation attributes and the consistency assumptions of semi-supervised learning are integrated together under the hidden Markov random field framework. The semi-supervised learning could overcome the problem of lacking of training data, and the sparse techniques catch the con-cept correlation more reasonably and efficiently, which improve the annotation performance and reduce the model complexity. Our method is evaluated on the TRECVID 2005 dataset, and conducts extensive comparative experiments with respect to 6 related methods.2. For the problem of visual content specific concept (ship) detection, we propose a ship detection scheme based on Contrast-Box filtering on the 2-D feature plane constructed with local intensity standard deviation. Taking the intensity standard deviation as detection feature could reach a consistent characterization for ships of both white and black polarity, and remove the brightness variances of sea background, meanwhile, reduce the problem to a reasonable scale. The Con- trast-Box filtering process could detect the target candidates on the detection fea-ture plane self-adaptively by exploiting the spatial structure information of the targets, and remove the false alarms caused by clouds, waves and ship tracks.3. For the noisy social-tagging results of the community-contributed multimedia visual contents, we propose a tag ranking algorithm based on visual content se-mantic relatedness. In this algorithm, the definition of semantic relatedness be-tween tag and visual content is formulated in probability based on Bayes'theo-rem, taking account to both the prior visual information related probability of tags and the relatedness likelihood between tag and specific visual content. Morever, because different visual features have different semantic gap size when representing different semantic contents, global and local features are fused to conduct the probability estimation more accurately. The proposed method is of semi-supervised in nature, and fullfills based on the internet data, and does not need any training data and the time cost of model training. This method is eva-luated on a large scale Flickr image dataset and the experimental results demon-strate that the visual content related tags could be distinguished from the contex-tual tags effectively.4. As a powerful supplement of keyword-based visual content retrieval scheme, we propose an interactive multimedia search scheme based on visual content color layout to help users get search results which are not only related in semantic but also in consistency of color layout. The color layout information is characterized by a novel feature in binary format which is compact in storage. The consistency definition between color layouts simultaneously considers the absolute, relative and contextual spatial distribution consistency of the colors. The consistency computation could be completed online through bit-wise operations. Moreover, a convenient interactive interface is presented to allow users to specify interest col-or layout flexibly. Extensive experiments are conducted on internet image search results to evaluate the proposed approach in every aspect, such as parameter sen-sitivity, time-space complexity, performance comparison, and user study.