The Research Of User Behavior Mining System And Implementation Based On The Massive Log

Since the late 20 th century, with the rapid development of the Internet, people's demand for the network is growing. So far, the Internet has become an indispensable part of life, the number of Internet users is growing. In such conditions, the user logs of users that produce in the PC terminal and mobile terminal are growing rapidly, and now it is difficult to estimate there are how many user behavior logs produced each day. These logs contain a large amount of informations, and are closely related to our daily lives.Based on the above situations, by reading a lot of references and research different sources of logs, the paper design of a user profile analyse system based on massive logs.The system includes: log through module, log preprocessing module, user's natural attribute mining module, user's app interests mining module and evaluation module. Log through module and log preprocessing module will get together logs from different sources to provide complete data for data mining. User's behavior mining module is the key part of the system. Evaluation module is the test of the mining results. In this paper,using the data mining process ideas as road map, massive logs as premise, Hadoop as experimental platform, HDFS as file system, MapReduce as programming framework,HBase as database to implement mining of user profile base on massive logs.This paper describes the design and implementation process of a user profile analyse system based on massive logs, and analyse detail design of each module. This paper firstly describes the project background and related technologies to provide technical support for the design of the system, and then introduces the needs of the project. Then detailed project design and implementation of the core module. Finally, this paper gives a brief summary describes the post-project areas for improvement.