Research And Application Of User Behavior Pattern Mining In Mobile Internet

As the internet has penetrated into all aspects of people's lives, the data generated in the network has been an explosion of growth. Now, we have entered the era of big data. Big data era has brought us both opportunities and challenges. The value of big data has guiding significance to the development of enterprises, even the country.Nowadays, there are many kinds of data traffic in the network needed to analysis and research, such as video traffic, audio traffic, transaction record and so on. In this paper, we select the DNS traffic to analysis. DNS protocol can be used in almost all of the network applications, thus the research has a great significance.We first introduce the background and importance of this thesis. Next,we explain the domain space, parsing process, message format of DNS protocol. These basic knowledge is the foundation for the design and implementation of the traffic collection system. Also, we illustrate the design principle, work process of the distributed processing platform,Spark, to provide a reliable platform for the traffic analysis and processing.This thesis mainly elaborates from three aspects. The first is the design and implementation of the DNS acquisition system. After the description of the deployment environment and basic architecture of the collection system,the thesis explains the implementation of DNS sub module in message processing and matching logic. The system is deployed in the campus and the core network. We analysis the distribution of the client and server traffic of the campus. For the traffic of core network, we choose the server which has the massive data to calculate two criteria to evaluate the QoS.Also, the subject proposed a system to detect the malicious domains. After sample collection and feature extraction, the system utilizes the classifiers to distinguish the malicious domains from the legitimate domains.This thesis makes use of the DNS traffic in the real network, and through the analysis and data mining, the research is very important to the network traffic study, especially the network security field.