Implicit Relevance Feedback Methods Based On Eye Tracking

In recent years, content based image retrieval technology has been developed rapidly. In order to improve the retrieval performance, relevance feedback technology which realized the users’interaction with the retrieval system has been introduced in retrieval system. Relevance feedback has become an indispensable module of image retrieval. How to use relevance feedback technology efficiently is still a hot spot in research.This paper proposes a relevance feedback method based on eye tracking, using the user’s eye movement data which could be regarded as more accurate feedback information to improve the performance of the retrieval system. The proposed implicit feedback method is implemented online and real-time, including three main modules:(1) the standard bag-of-words image retrieval module;(2) eye tracking and data processing module;(3) query expansion module. According to feedback information, this paper has designed two kinds of implicit relevance feedback methods based on eye tracking. Specifically, the feedback method based on concept image applies entire images as research object. The extended query image is constructed by a combination of images according to their fixation duration, therefore improving retrieval results. Feedback method based on semantic space works on interested area of images. This method creates new query by combining interested areas in different images according to their fixation duration, and then uses the visual words to improve the retrieval results. To10participants, with the feedback method based on concept image, there is70%and50%participants who are satisfied with the result of feedback in Oxford building database and UKbench database respectively, and there is30%and60%of the retrieval results that are improved in the average precision accordingly. For the same participants, with the feedback method based on semantic space, there is70%and60%participants who are satisfied with the result of feedback in Oxford building database and UKbench database respectively, and there is70%and80%of the retrieval results that are improved in the average precision accordingly.