Ensemble Of Multiple Descriptors For Automatic Image Annotation

With the development of digital equipment and computer network, the number of digital images is growing dramatically. In order to manage the large volumes of digital images automatically, various image retrieval techniques, especially image automatically annotation algorithms, draw a lot of attentions in recent years. Aiming at handling the problem of multiple annotations and"semantic gap"of images, this paper investigates how to utilize multiple visual descriptors to improve the performance of automatically image annotation.The major contents and contributions of this paper are:(1) In order to cope with the problem of lacking linkages between images regions and label keywords, this paper employs a Normalized Cuts algorithm to segment each image into image regions before annotation, and then uses an image annotation strategy on each image region respectively, so as to create one-VS-one linkages between image regions and label keywords.(2) As currently only limited labeled samples are available while a large amount of unlabeled samples exist in a given image annotation system, this paper adopts the idea of semi-supervised learning to construct learning models. The model utilizes both labeled and unlabeled samples for training, and then uses visual descriptors to measure the distance between unlabeled images and labeled keywords, so as to choose the appropriate keywords for the unlabeled image.(3)As most available automatic image annotation algorithms use only a single classifier to predict keywords, the accuracy of these algorithms are relatively low. This paper proposes a novel algorithm, EMDAIA, for automatic image annotation. EMDAIA regards the image annotation task as an image classification task, and integrates the classification results of multiple classifiers, i.e., selects the keywords with the maximum probability to label the images. Experimental results on LabelMe dataset demonstrate that EMDAIA achieves approximately 10% improvement in accuracy over the single descriptor approachs.(4) The paper also proposes a ROIAIA algorithm for automatic image annotation. Before annotating an unlabeled image, ROIAIA firstly uses the Itti model to extract the region of interest, and then annotates the interest regions by keywords. Experimental results show that, ROIAIA can effectively reduce the impact of the secondary regions, and greatly improve the speed of automatic image annotation.(5) In order to evaluate the efficiency of annotation by different feature subsets, this paper extracts several of features, and chooses 10 different feature subsets for experiments. Experimental results on the PASCAL VOC 2008 database show that using ensemble of multiple descriptors can improve the accuracy of annotation. It achieves the best annotation performance while integrating 8 descriptors. Afterwards, its performance will degrade while keeping increasing the use of descriptors.