Analysis And Distributed Processing Of Large-Scale Mobile Internet Flow

With the development of mobile Internet, mobile Internet gradually over the PC platform and become the main entrance of the Internet. Because of their mobility, mobile Internet became a information exchange center of consulting, social, trade, games and so on all kinds of industry. Accordingly, the mobile Internet market is gradually expanding. The rapid development of mobile Internet, making the study of the characteristics of the mobile Internet, and found the difference between traditional wired networks and mobile Internet become an urgent problem. Cable network has proven to be a power-law distribution characteristic of a complex network, whether mobile Internet has related properties of complex networks, or other features that can further be applied in traffic identification and other areas need to be studied. One major feature of mobile Internet applications is that mobile Internet has rich variety of applications. The role of application in mobile internet and the relations of traffic and applications and relations with all network users need to be study.But the rapid growth of mobile phone users and increasing use of mobile Internet brought a large number of mobile Internet data. Daily flow data record in GB and even TB rate of growth. How to deal with large-scale data and digging out useful information become the problem which must be faced. However, distributed processing technology offer cheap and reliable solution ideas for those problems. Traditional machine learning algorithms have proven that it can play a huge role in many areas. But for facing of a massive amount of data the mobile Internet, traditional machine learning algorithms require a distributed implementation that can be applied to existing data. Traffic identification is one of the hot research in study of network. The traditional traffic identification is achieved by analyzing packet data, without concerning relations between the flows. To find the relationship between traffic by analysis and mining of mobile Internet data, and propose new ideas for traffic identification is one of the main theme of this study.In this paper, for that the characteristics of complex networks for mobile Internet properties is unknown, this paper analyze the characteristics of the mobile Internet and find the contribution of mutual communications of users to flow and the server’s concentration. DBSCAN distributed algorithm has the problem of excessive traffic, so that a optimization method is proposed to reduce the traffic while also reduce the amount of computation under the premise of no influence on clustering results. For that traditional traffic flow identification technology does not concern of relations in flow, clustering feature and high degree of purity of servers is found by the analysis of flow graph. Based on the these characteristics of servers, traffic identification problem can be converted to the identification of the server. Server clustering and identification are realized by clustering algorithm, and the results show a high accuracy rate.