Web Image Semantic Analysis And Automatic Tagging Study

Various types of information are usually available for Web images, such as the basic visual features (color, texture, shape etc.) and the associated textual features. It is well known that the semantics of Web images are well correlated with these associated informations. The previous research work demonstrate that there exist a huge "semantic gap" between the low-level visual space and the upper-level semantic space of images, and this results in the poor performance of visual content based image semantic annotation. The associated textual space is closer to the semantic space of Web images than the visual space, so they can be well used to infer the semantics of Web images. However, the relation between the semantic contents of Web images and their features is very intricate and various, and different annotation keywords or Web images usually correspond to different semantic distributions. Most previous work either regard the associated texts as a whole, or assign fixed weights to different types of associated texts only according to some prior knowledge or heuristics, and the performance of Web image semantic annotation is still need to be further improved.This paper studies the intricacy characteristic of semantic distribution of Web images. Based on the adaptive learning idea, several Web image annotation methods with good performance are proposed.The main works of this paper are as follows:1. Position weights adaptive learning of the associated texts based automatic Web image semantic annotation approach is proposed: we use the basic expansion method to further consider the semantic contributions of the high order structure relation between different types of the associated texts, and propose a piecewise penalized weighted regression model to adaptively model the Web image's semantic distribution on the corresponding associated texts. The experimental results on a real world benchmark show that our method can improve the annotation performance greatly.2. The adaptive model based automatic Web image semantic annotation approach is proposed: this method further leverages the visual features and the prior knowledge to improve the annotation performance on the basis of position weights adaptive learning of the associated texts based automatic web image annotation approach. To incorporate the contribution of prior knowledge, we propose a constrained piecewise penalized weighted regression mode to adaptively model the Web image's semantic distribution on the corresponding associated texts. The experimental results on a real world benchmark show that our method our method can improve the annotation performance greatly.3. The conditional random field model based automatic Web image semantic annotation approach is proposed: this method provides a unified annotation framework to combine different types of features of Web images to improve annotation performance. We further explore the manually tags resources of Flickr to improve the estimation of semantic correlation between annotation keywords. The experimental results on a real world benchmark show that our method outperforms the state-of-the-art Web image annotation method.4. A Web multimedia information retrieval prototype system is proposed: wepresent a novel Web multimedia information retrieval prototype system-PictureBook, which combines text and image retrieval using techniques of search results clustering, multiple document summarization and Web image semantics analysis. Particularly, audience can interactively investigate the effect of the combined text and image search results in Web information searching and knowledge acquisition.