Research On Image Annotation Based On Multiple Features

With the development of Internet and Multimedia Information technology, the image data which can be accessed and obtained in people’s daily life is increasing all the time. The wide application of Mobile Internet technology allows people to upload and share images anytime. Due to the nature of the image date itself, the image data could not be effectively analyzed and managed as the text data. As a result, the image data may not be used effectively in information retrieval. Nonetheless, crowdsourcing may cause huge cost in money and high cost in time. And crowdsourcing may lead to different image annotation results because it is based on personal understanding of images and concepts. In addition, there different languages in text while the image is crossing-language. Annotating the image directly could solve the problem effectively. Otherwise, the users would not get the very image they want by searching the keyword of text. In this case, how to realize the image content by automatic image annotation for analysis, management and retrieval becomes a hot issue in recent years.Image annotation is an important part in image retrieval. Image annotation gathers a variety of new technologies such as computer vision, pattern recognition and image processing. In this thesis, in order to solve the problem of automatic image annotation, we make a study of different image features and try to describe the content of images by a variety of image feature descriptors. Taking multiple image features into account, we get a preliminary result of automatic image annotation. The main contents of this thesis are as follows:(1) In this thesis, we analyzes the overall process of automatic image annotation and test several image features extracted from test image set for study. We use different image features, such as Colorhist, GIST and SIFT features, to describe the content of image from different aspects.(2) In training period, we use the Multi-Modality Learning approach in machine learning to build the relationship between different features and corresponding images and establish the link between image feature vectors and image content. As the same concept in different images may have similar or identical feature vectors, we separate the image features into different modalities. We train only one concept each time and we can obtain the characteristic of the modality to the concept, then we could obtain the classifier to the corresponding concept. We try to use one-hidden-layer and multi-hidden-layer artificial neural networks to annotate the images.(3) There are some large data for which with/without nonlinear mappings gives similar performances. Without using kernels, one can quickly train a much larger set via a linear classifier. As a result, we try to annotate images with LIBLINEAR.(4) We extract image features with VGG model by using Caffe and train the ranking scores with multilayer perceptron. We fulfill the annotation and retrieval in both directions of sentences and images, and an online annotation and retrieval system.