A Semi-Automatic Image Annotation System Based On Semantics And Contents

Recent years have seen a rapid increase of the size of digital image collections. Everyday, both military and civilian equipment generates gigabytes of images. However, we can not access to or make use of the information unless it is organized so as to allow efficient browsing, searching and retrieval. Generally, there are two kinds of image retrieval: semantic-based and content-based. Semantic-based image retrieval uses the additional information of an image such as title, keywords and some descriptors to represent and index the image and users can use some text-based approach to search the images they want. For an example, a user first types in a keyword 'house', then system returns all the images annotated with 'house'. However, as most semantic-based image retrieval systems require manual annotation of images, obviously this kind of system is not efficient. Content-based image retrieval uses the internal information of an image such as color, texture and shape to represent and index the image and users can use some feature vector comparison methods to search the images they want. For an example, a user first provides a sketch of house and system returns all the images that have similar feature vectors with this image. However, as the limitation of the recent content-based technology, this kind of system still can not easily give users an accurate result.In this thesis, we will propose a novel semi-automatic image annotation framework. Here is a user scenario: a user first types in keywords of the image he wants, system then uses the semantic-base approach to return all the images matching this keyword. According to his requirements, this user gives some feedback to the system to show which images he thinks are relevant to the image he wants. To these images, system does some content-based retrieval to find more relevant images. For each relevant image, system will reweight its corresponding annotation information. This describes a brief progress of a semi-automatic image annotation system. The coverage and quality of image annotation in such a system is improved progressively as the cycle of search and feedback increased. The strategy of semi-automatic image annotation is better than semantic-based image retrieval in terms of efficiency and better than content-based image retrieval in terms of accuracy. We have used this strategy in our Image Annotation 1.0 system and got a satisfactory result.