Research Of Focus Search Engine And Its Application On Community Informatization

The concept of "Cloud computing" is proposed by Google in2006. This business model provides a totally new idea for the industry and academia. Dongfeng yuan, professor at the school of information science and engineering of shandong university, whose team quickly seized this opportunity, launched a new cloud-based information model with in-depth study and achieved initial results. This team has been supported by two major projects of independent innovation achievements of Shandong Province. This paper is derived from the second major projects,"low-cost, low-power, high-reliability embedded terminal and information service platform"(2010ZHZX1A1001) project.Under the trend of urbanization throughout the country, in view of the countryside being built into a community, scale operation and collective economy has been started. Rural reconstruction work has achieved rapid development in Shandong province, the selected pilot areas of this major project is a typically rural community transformed by countryside, the community information construction has also become a very important part. In the national development strategy from2006to2020, the construction of community information is listed as one of the strategic priorities of the development of China’s information technology. In this context, the project team has launched a key technology research of information technology, a new information mode of "cloud computing server and broadband network and thin client" has been proposed. The project team developed embedded architecture based thin clients, cost and power consumption are reduced to a very low level. A cloud computing server clusters is been deployed, a few kind of applications have also been developed based on a survey to the community users. This model is used to replace the traditional PC as the core of the information road. A large-scale pilot demonstration has been launcheded, and good results have been achieved.To satisfy the requirements of the target users combined with the characteristics of the new community information mode; this paper designs a focus search engine for taobao shopping, to provide users with a convenient shopping search and recommendation. Considering the variety kinds of goods on taobao website, a generic commodity model has been designed and implemented. This system has two modules: web crawler and searcher. Web crawler module realizes crawling information from website and establishing index file and storage information into database. Searcher module realizes keywords query, index files queries and database queries, providing search results and information recommended to users.In crawler module, in order to cope with the the crawl efficiency problem of the huge amounts of data, a distribute network crawler based on hadoop has been realized with Java language. Firstly, the distribute environment has been built up under ubuntu9.10operation system. Then distribute crawler based on hadoop is been designed, establishing of index file has been realized through designing data storage strategy; information extraction method has been designed and information storage strategy has been realized.In the search module, Browser-based search function has been deployed. The search procedure is J2EE project. The system firstly realiezes runtime environment configuration function to set system params, then develops a user search interface to search index files and database query. Users can choose to sort result by price, or view product details query, commodity prices, title, description, price curve, and it’s similar price range goods.