Manual Evaluation Methods To Evaluate The Online Store Retrieval System

With the internet applications boosting, Internet-based online stores are more and more accepted.And its growth rate far better than the traditional offline distributionchannels. Based on the large number of goods'categories andquantity, online stores are most liked to provide buyer experience with searching functions. Therefore, how to improve the quality of search results, how to help users find the satisfaction goods is an important challenge for online stores. Tosolve this problem, firstly need to establish a reliable evaluation system to describe search results'quality. Thencontinuously improves IR system with analyzing the evaluation results.Crowd source evaluationmethod, human judgment, is elaborated as one workable methodology for online store IR evaluation modules. Golden Set usage, identified most valuable item attributes and cost control are discussed as human judgment system fundamental requirements; Taking advantage on human judgment result output data with IR common statistic method; IRR experiments show AC1 does well for filter the noise in human judgment result. In conclusion, human judgment, with capability of user motivation capturing, can be used as critical method for online store IR system.