Pattern Mining And Analysis Based On Telecom Data

With the rapid development of information society, the data has been expanded day by day, and how to convert these data into useful information and knowledge will be the essential solving problem to the data mining area. Pattern mining is an important part of data mining, through which we can acquire the general property of the original data and can infer and predict information by the property detected, and can obtain useful information ultimately. Telecom data is one of the important information source, detecting and analyzing the pattern on telecom network, can help the operator to establish the corresponding marketing strategy, but also can assist analyzing terrorist organizations. Telecom network is a typical social network, and using social network analysis method to detect and analyze the telecom network can help us obtain information and knowledge more accurately. This article aims to detect as many patterns as possible and provide the analysis method to these patterns. It also provides a process workflow based on Hadoop, as well as a prototype system based on this workflow.First of all, this paper provides a process procedure for data mining and analysis based on Hadoop distributed computing environment. And the process includes ETL, data mining engine, effect evaluation and knowledge representation.For the pattern mining, as users do not know which kinds of patterns in data are interesting, we need to detect different patterns as many as possible. In this article, based on telecom network, our detected patterns are mainly related to periodical pattern, outlier pattern, spammer pattern, structure correlation pattern, small connected component pattern and anomaly pattern based on the pattern we find above. Through pattern mining, we obtain the targets which are the objects needed to be analyzed later. In this article, the telecom network analysis includes egocentric network analysis, community detection analysis, visual analysis and so on. We can discover, explore and identify these found patterns in detail through our analysis. And more, we can get new information and knowledge.Finally, we develop a prototype system based on a real project. This prototype system can better reflect our mining and analysis process and combines a part of our detected patterns and pattern analysis method, which is a better reflection for the combination of research and application.