Design And Implementation Of Internet Users Behavior Acquisition And Analysis System

Nowadays,with the rapid development of Internet business,the personalized needs of users become more intense then ever.Thus,by analyzing user behavior in order to provide customized and personalized service has become the key for the development of many enterprises.In the process of visiting web pages on the Internet,usually the users will leave large amount of traces of their behaviors.A deep data mining will be utilized after the user behavior information is collected,filterd.By this way,people can make a better understand of the users' expectations and aspirations,by refering to which,the enterprises can make better decisions.Furthermore,since most of the users have a strict standard for responding time,the key problem to be solved is how to deal with real-time analysis of user behavior.This subject is based on research project of "XiNiu data platform",which is I'm participated in during the internship in the department of Neusoft XiKang,the main research of this topic goes around the behavior of users to access XiKang networks and the index of the medical report of network users to expanded.The core parts of study in this thesis are briefly described as follows:First,this thesis introduces the basic concepts of data mining as a start point,after which,gives a description of some basic approaches and function of data mining.Since this system mainly focuses on Web log mining,thus it also makes a introduction of some basic concepts concerning Web mining,log mining as well as some basic features and knowledge of user behavior analysis.Then,make a general introduction to basic conception of data warehouse and ETL technology,make a detail description of the details of Storm Real-time processsing.Secondly,completed the overall architecture design of XiNiu data platform,this architecture consists of Storm based real-time analysis and off-line analysis based data warehouse.Then various functional modules are carefully designed for the data platform in this system,including the module of data acquisition,data processing,data analysis and data display.Finally,implement the data platform based on the Storm and Oracle,after which,test all the functional modules.Then,make off-line analysis of user behavior based on data warehouse,make real-time analysis of user behavior which is based on Storm.At last,display the result.By building the XiNiu data platform,the visualization display of off-line and real-time result graphs,that can effectively to analysis and mining the behavior of Internet users,in which way,some instructive information can be gained concerning XiKang network,thus provide useful information for the decision makers.