Research On Visual Information Annotation With Machine Learning Techniques

With the advances in storage devices,networks,and compression techniques,large-scale visual data become available to more and more ordinary users.How to effectively organize, represent,manage and retrieve these data becomes a challenging task in both research and industry.To achieve this goal,visual information annotation has attracted more and more attention.The most intuitive approach to accomplishing this task is manual annotation.However, manual annotation is a labor-intensive and time-consuming process,and it can hardly be applied for large-scale data set or concept set.Thus,learning-based visual information annotation becomes an alternative method,in this thesis,we propose several learning-based visual information annotation methods,which aim to obtain accurate annotation results automatically.The main contributions are illustrated as follows:1.We propose a novel visual information annotation framework that accomplishs annotation with discovering concepts' properties.Based on the framework,two new annotation refinement algorithms are developed,which aim to improve the annotation by leveraging statistical correlation and semantic correlation among the concepts,respectively. Compare with conventional annotation methods that treat concepts independently,our approaches can achieve superior performance on visual information annotation.2.Semi-supervised learning methods,which attempt to trackle training data insufficiency problem,are widely adopted for visual infroamtion annotation.Conventional semi-supervised learning methods predominantly foucs on single concept learning problem.However,visual information annotation is essentially a multi-concept learning task.In this paper,we propose an innovative semi-supervised multi-concept learning framework.This framework is characterized by siumultaneously exploiting the inherent correlation among multiple concepts and the annotaiton consistency over the sample graph.Based on the proposed framework,we further develop two novel semi-supervised multi-concept leanring algorithms.We apply them to visual information annotation and obtain superior performance compared to the state-of-the-art semi-supervised approaches.3.Recently,multi-instance learning technique,which attemps to trackle data ambiguity problem,has been utilized for visual information annotation.Traditional multi-instance learning algorithms mainly focus on single concept learning problem.However,visual information annotation is essentially a multi-concept learning problem.In this paper,we propose an innovative multi-instance multi-concept learning method which simultaneously captures both the connections between semantic concepts and regions,as well as the correlation among multiple concepts.Moreover,the proposed approach is also able to capture other dependencies among the concepts,such as the spatial relations. We apply the propose approach to image annotation and report superior performance comparped to key existing methods.Visual information annotation is closely related with many different domains,such as machine learning,computer vision and cognitive science.We also hope that our work can provide several inspirations or methods for these communities.