The Study Of Reranking Method For Image Retrieval Based On Eye Tracking

CBIR technique has attracted much attention in recent years, because of the existence of semantic gap, initial retrieval performance is not satisfactory. In order to improve the retrieval performance, some reranking methods have been proposed, among which interactive reranking method has become a hot issue.This paper puts forward a reranking method for image retrieval based on eye tracking. The eye-movement data is captured by an eye tracker when the user watches the result images, then the eye-movement data will be analyzed in real-time. Some user-interested information will be found out, and then we reconstruct these information to formulate a new query and retrieve the database again to obtain more satisfactory result images. In this paper, the image retrieval system with reranking method based on eye tracking is mainly divided into two modules:(1) BoW image retrieval module;(2) the eye tracking and reranking module. The BoW image retrieval module mainly completes five processes:extracting image feature, clustering visual words, quantizing feature points, establishing inverted index and retrieval service; while the eye tracking and reranking module mainly include three processes:eye movement data recording, real-time analysis and new query generation.When a user watches different kinds of images, the eye movement data is also analyzed in this paper. The author finds that when a user observes an image, the fixations are more preferred to locate in the central part of the image, meanwhile a fixation model is built to adapt to the reranking method.For the proposed approach, this paper uses the Oxford architecture data set, containing5062images, as test data set to evaluate retrieval performance before and after reraking. We invited24students as subjects to participate in the experiment, each subject watched12query images. According to the user’s eye-movement data and retrieval performance, the reranking method proposed in this paper can improve the retrieval performance, and the average precision can be increased by13.57%.