Research And Implementation Of Web Log Mining System

With the rapid development of Internet technology, data on the Internet is in explosive growth at incredible speed, the popularity of Web2.0let each participant of the Internet become not only information consumers but also the publisher of information, Era of massive data has arrived.In the face of massive data, users always want to use one of the most lightweight, fastest, most direct way to get to the desired content, the search engines just to meet the needs of all users, more and more people tend to use search engines as their entrance to access knowledge and information on the Internet. The search engine server logs all user access behaviors, analysis of these log data can get a lot of in-depth knowledge, lead Web technologies to further improvement, and strengthen the search engine’s intelligent processing, self-learning feature. To this end, Web log mining (also known as Web Usage Mining) technology emerges at the right moment, and become a hot spot of the present Internet research.This paper uses academic search engine log files as its research subjects, and seeks to bring more help to Science and Technology Information Service. First of all, this paper describes the history and current development research of Web Log Mining, and on this basis further expand the description of the various techniques involved in this study. After that, this paper elaborates all the details to the prototype system design and implementation with four major system functions:Data Preprocessing (including Data Cleaning, Data Correction, Data Compression), Pattern Piscovery (including User Identification, Session Segmentation, Content Query Discovery), Pattern Analysis (including Statistical Analysis, Sequence Analysis, etc.), and the Query Recommended based on Semantic Association.Finally, the system provides user-friendly interface, and convert the results to visual charts for user-friendly analysis. In addition, this paper also puts forward the idea to combine Web Log Mining system with Author Networks and Keywords Networks, in order to make the system function more perfect and provide users with more technology information services.