Design And Implementation Of User Behavior Analysis System Based On The Hadoop

With the great development of information technology, the computer bring great convenience for People’s Daily life and work, at the same time, the user behavior data generated by the computer itself increase at exponential scale. How to analyze of users genuine needs and preferences from these huge amounts of data, is a very valuable research.The main works of this article are:(1)Establish the data cleaning algorithm, behavior log and configuration information parse algorithm to realize data preprocessing. Filter the data which have error key words. Set data completion rules and fill the key words missing data. Calculate and compare the MD5 value of user behavior data, deletes multiple repeated data such as data cleaning, and use the JSON parsing technology on the interpretation of logs and configuration information, organize user behavior analysis data needed.(2)Graphs calculation based on the Hadoop framework, user behavior rule mining algorithm are designed and implemented on the user behavior analysis, find out the number and length of users to use the USB, ODD and Card Reader, the average brightness adjusting times and different interactive equipment input frequency, mode using the length and the characteristics of the switching frequency, etc to provide data support for producers optimize devices.(3)Reduce dimensions of user behavior regular data based on data filtering interactive technology, and reduce feature space. Using the histogram display intuitively abstractly regularity of user behavior data, realize the data visualization analysis, allows the user behaviors characteristics clearer. At the same time, in the system to add permissions management, users can access and only access authorized resources, improve the safety of the system.