Web Image Ranking And Mutual Summarization

With the rapid development of the Internet, with the improvement of mobile phone, digital camera, tablet computer, etc., more and more images, videos and other web media are pouring into the Internet and continuous spreading. Facing such a large-scale Web image data, how to organization and retrieval them is a important problem in both academia and industry.This paper mainly focus on three image retrieval sub-problems:(1) image ranking and re-ranking,(2) action recognition and retrieval in still images,(3) mutual summarization of image and text.1. Image ranking and re-ranking: ranking is an important problem of information retrieval. Ranking the results user wanted in the top positions is the focus of research. This paper makes use of the Learning To Rank (LTR) ideas, combining the characteristics of image retrieval and specific problems, introduces a method: learning to rank for web image retrieval based on Genetic Programming (GP). The model employs text, visual and link information of web image, utilizes MAP as the fitness function to solve the optimization problem. Traditional image ranking models just use the text information and neglect the visual information of web images. Especially the retrieval results lost the visual diversity, which is unfavorable for the users to find the desired results as soon as possible. To solve this problem, this paper proposes Dual-Ranking model to improve the ranking diversity of image retrieval. This model formulates clustering as a constrained multi-objective optimization problem. The framework of Dual-Rank is composed of Inter-cluster Rank and Intra-cluster Rank to rank clusters and images respectively.2. Action recognition and retrieval in still images:for the action or event queries, the traditional methods just focus the text matching technology, and no consideration for the high-level semantic understanding of web images. In order to solve this problem, this paper designs a action recognition and classification model based on multiple kernel learning, learns an optimal combination of histogram intersection kernels, each of which captures a state-of-the-art feature channel. The model ensures the premise of semantic similarity, enhances the visual similarity of results.3. Image-text mutual summarization: using some sentences or even text to describe the sophisticated semantic information of web images is a meaningful work. It can further improve the richness of image annotation. It will be better than simply keywords or tags labeling images, because sentences and text contain more abundant information. Therefore, this paper proposes a simple method to generate sentences automatically to describe what is happening in a still image. The sentences are concise, just including the main ingredients:subject, predicate, object and the object complement, etc.For some web images, they often come with articles, such as the News. Therefore, how to use such rich text information to summarize the images with text information? Also, how to choose a picture to visualize illustrate a text? This is an interesting problem. To tackle this problem, this paper introduces a image-text mutual summarization model, to summarize images with sentences or text, and to visualize text with images. This paper divides the web image text data space into three subspaces, namely pure image space (PIS), pure text space (PTS) and image-text joint space (ITJS). For summarizing images by sentence issue, map images from PIS to ITJS via image classification. For text visualization problem, we map texts from PTS to ITJS via text categorization models and generate the visualization by choosing the semantic related images from ITJS, where the selected images are ranked by confidence.In view of the different problems, this paper designs different experiments on both the dataset we collected and some other open dataset. By comparing the results with some other methods in terms of Accuracy, P@n, Map, PR curve, etc., our methods perform very well.