Research On Image Classification With User Annotation

With the progress in cloud computing, image transmission and computer networks techniques, large-scale visual data including web image and video act a key role in web development. How to efficiently organize, represent and assign these data turns out to be a challenging research track. To achieve this goal, web image classification has attracted more attention.The most intuitive approach to accomplishing this task is to represent image visual content. To better translate image content, we should be more focus on image feature extraction. Two types of visual features are widely used: global and local feature, which characterize the whole image and particular regions of the image, respectively. As compared to global feature, local feature attracts much more attention recently since it is robust to clutter and occlusion. In this thesis, we propose several web image classification methods based on local features and user interaction, which aims to obtain better classification results. The main contributions are illustrated as follows:1. Local image feature has received increasing attention in various applications, such as web image classification and search. The process of local feature extraction consists of two main steps: interest point detection and local feature description. A wealth of interest point detectors have been proposed in last decades. Most of them measure pixel-wise differences in image intensity or color. Recently, a new type of interest point detector has been developed, which incorporates histogram-based representation into the process of interest point detection. In this paper, we evaluate this histogram-based interest point detector in the context of web image classification and search, as well as compare it against typical pixel-based detectors and heuristic grid-based detector. The evaluation is performed on two web image datasets: NUS-WIDE-OBJECT and MIRFLICKR-25000 datasets. The experimental results demonstrate that the histogram-based interest point detector outperforms the pixel based and grid-based detectors in both web image classification and search tasks.2. Flickr groups are self-organized communities to share photos and conversations with common interest and have gained massive popularity. Users in Flickr have to manually assign each image to the appropriated group. Manual assignment requires users to be familiar with existing images in each group and it is so intractable and tedious that prohibits users from exploiting the relevant groups. For solution to the problem, group suggestion has recently become an important research topic, which aims to suggest groups to users for a specific image. Existing works illustrate group suggestion as the automatic group prediction problem with a purpose of predicting the groups of each image automatically. However, the automatic prediction is still not accurate enough. In this paper, we propose a semi-automatic group suggestion approach with Human-in-the-Loop. Once obtaining user's feedbacks on the representative images selected from user's image collection, we infer the groups of remaining images through group propagation over multiple sparse graphs of the images. We conduct experiment on 15 Flickr groups with 127,500 images. The experimental results demonstrate the proposed framework is able to provide accurate group suggestions with a quite small amount of user effort.3. Recently, interest in personalized tag recommendation is sparked within the research community. In order to predict or recommend tags for a specific user precisely, the recommender should first distinguish user's preference and interests for each individual. In this algorithm, the definition of semantic relatedness is formulated in probability based on Bayes theorem, considering both the prior visual information related probability of tags and the relatedness likelihood between tag and specific visual content. Global and local features are fused to conduct the probability estimation more accurately. Our method is evaluated on a large scale Flickr image dataset and the experimental results improve the effectiveness and efficiency in tag recommendation.