Research And Application Of Comparison Shopping Engines

With the rapidly growing numbers of service providers for e-commerce sites and products in the field of online shopping, consumers are faced with more puzzles. How to find websites and items that suit their own needs? Where are cheaper products? How to get the most cost-effective products and services? These questions all make the consumers feel confused. Comparison shopping is a kind of shopping agent software that takes information obtaining technology. This kind of software compares prices of commodities, performance, delivering methods and services to provide consumers commodities information of different online merchants by collecting information about online merchants. Therefore it is a way of shopping that can save not only your time and effort but also your money, which has obtain good judges from most e-commerce consumers.This thesis mainly studies how to crawl and analyze the web data. Crawling the web data is done by spider. Web data analysis refers to extracting structured information from the crawled web pages. In the part of web data crawling, this thesis improves the traditional framework when spider crawls web data in general search engine. According to the characteristics of the professional search engine, this thesis proposes an optimized web page data crawling framework. That is,"url extracting intervention" is added when spider crawl url links, thus improving the efficiency of specialized search engines crawling the web data. In the part of web data analysis, this thesis completed a well-extended web data extracting module.In order to solve the speed contradiction between web data crawling and analysis, this thesis designs a kind of producer-consumer model. That is, creating a kind of web page data caching mechanism between data capturing and data processing, so as to improve the overall performance of the search engine.In addition, currently the domestic comparison shopping websites only provide consumers price comparison between products, lack of comparison of consumers’ remarks about different shopping websites, resulting in that the information reference is still not sufficient. So this thesis extracts the positive feedback percentages of products when processing the web page data. Although price is the main factor that affects consumers’shopping choice currently, the remarks of products are acting a bigger and bigger role when people make their shopping choice with the development of online shopping.At last, this thesis designs and implements a prototype of comparison shopping search engine system taking the mobile phone datas in Jingdong electronic mall and amazon electronic mall as the example of data crawling, which can not only provide comparison of price, but also comparison of evaluations, in order to provide consumers better shopping experience.