The Design And Implementation Of An E-commerce Website Search Logs Analysis System

With the rapid development of the Internet and the rapid increase in the number of websites,the website's of the user's competition has become increasingly fierce.In order to attract and retain customers better,website operators need to understand the behavior of web users for better,research and analysis of the log search engine has become a major method to acquire the effective data of user behavior.Based on this,in order to catch the actual needs of website users for better and understand the user intention,this thesis designs and implements a website search log analysis system,so as to help the website to better serve customers,and realize the web site rapid development.Different website search engines have different target groups,the research object of this thesis is an e-commerce industry website search logs,through the establishment of log analysis system to understand user behavior patterns,and tap the potential needs.The biggest difficulty in the design of the system is how to search the massive log data,and realize the high speed and accuracy of the search.The main research contents of this thesis are as follows:1,Search log collection format use the NCSA extended log format,web page analysis section use the label records,log collection system use the Apache and Flume log collection system,this make the web log collection efficiently,accurately and timely,and reduce the development and testing of the pressure and burden,and reduce the risk also.Page statistics make the log analysis simply and accurately by add labels,reducing the burden of log analysis.2,Log analysis use the distributed processing platform of Hadoop,this thesis development mainly based on the key technologies of distributed processing of the HDFS file storage and Map/Reduce,the implement process of the log analysis is described and analyzed in detail,through the use of Hadoop to solve the problem of the timeliness and accuracy for massive log analysis,the code develop is very simple,the difficulty of development is reduce greatly,greatly improve the efficiency of the project development.3,The thesis design and realize the user behavior analysis model and user information quality scoring model,through these two model,we can know the user'sweb browsing preference and the quality of user information,and we can know the keywords correlation information too.The thesis established user preference browsing model and information clustering model to provide data support for the information aggregation and personalized search.Finally,through the analysis of the results of the on-line system after operate for two weeks,the search system sort the search results again by the result of the log analysis,the improve of use effect is very good,the system has achieved the desired goal.