Research On Automatic Evaluation Technology For Search Engines Based On User Behaviors

The emergence of search engine completely changed the way people access to information, users can acquire comprehensive information from the vast internet ocean quickly and accurately. How to improve the per-formance of search engine which can better satisfy users’needs, has been one of the research focus of information retrieval field, and the core of search engine improvement is the evaluation of search engine.With the increasing of internet information and the development of computer technology, the traditional evaluation method which based on manual can’t meet the practical needs, improve the automatic evaluation is imperative. Introducing user behavior analysis to achieve the automatic evaluation is the main direction of research recent years. In addition, the ultimate goal of search engine is to better satisfy users, so understanding users’behavior habit and their views are significant for improve search engine evaluation. For these reasons, we launched a questionnaire about domestic users’behavior habit when they using search engine, through result data analysis, we summarized many important conclusions.Based on the results of the survey, this paper proposed an automatic method of search engine evaluation based on user behavior analysis, and constructed an automatic search engine evaluation model. The model consists three modules:firstly, we extract queries and corresponding in-formation about user behavior from web search logs automatically; then we annotate the retrieval results based on user behavior analysis; finally in order to verify the effectiveness of the method, we compared the dif-ference between the result and standard answer through traditional evalu-ate metrics, such MAP (Mean Average Precision) and NDCG (Normalize Discounted the Cumulative for Gain), the standard answers were obtained artificially. Experimental results showed that our method based on user behavior has higher accuracy than traditional method that based on CTR (Click-through Rate). Besides, we introduced a new element repetition rate to NDCG, and proposed an improved metric for retrieval results evaluation. Investigatory results showed that the improved metric can be more accurate and have more realistic sense.