Study On The Evaluation Index System For CBIR Systems From Users' Perspective

CBIR, Content-Based Image Retrieval, is a new kind of image retrieval technique after the emerging of the text-based image retrieval technique (TBIR), which has already been not able to meet the extensive needs of the user. As the CBIR technology is becoming increasingly sophisticated and more commercial & scientific CBIR systems are developed and established, it turns to be an urgent trend to carry out comprehensive evaluation studies on CBIR systems. The current evaluation research of CBIR system is still in the initial stage, and the topics are mainly focused on the technical aspects and performance evaluation index research. At the same time, the evaluation methods mainly follows that of the traditional IR systems, being obviously lack of particular research on the specific information form of images.The paper attempts to discuss the evaluation scope and contents of CBIR systems from the users'perspective by reviewing the status and achievements of the evaluation on traditional IR systems and CBIR systems, as well as the user's information seeking behaviors in their search of text information and images. Then, based on the theoretic exploration, the author established a comprehensive evaluation index system for CIBR systems from users'perspective. The evaluation system evaluates the overall performance from four aspects, including the database quality, ease of use, retrieval functions and search results. By further analysis, the four indexes are broken into 14 indexes of the 2nd tier and 59 indexes of the 3rd tier. Based on the evaluation system, these indexes are quantified by questionnaire and analytic hierarchy process (AHP) methods to construct a quantified evaluation matrix, so that the evaluation of CBIR systems could be more systematic, feasible and exercisable. Afterwards, two well-known CBIR systems accessible on the web are chosen for empirical study and their overall performance are evaluated to verify the applicability of the model. In the end of the paper, based on these research results, some recommendations are put forward to improve the overall performance of a CBIR system and to increase user's satisfaction from the technical level, hoping that it will ever benefit to the design, development and establishment of CBIR systems, as well as the further improvement of CBIR technology.