Researches On Data Mining Based Alarm Correlation Analysis In Communication Networks

Intelligent network fault diagnosis and location is a tendency of modern network management. The recent global expansion in the demand for communications services has resulted in the considerable growth of networks in terms of size, complexity and bandwidth. Nowadays, communication network turns to be much more complex. When a network problem or failure occurs, it is possible that a very large number of alarm messages are generated. As a result, it is possible to analyze the correlation of alarms in order to locate the source of fault effectively. Traditional expert system has difficulty in getting knowledge when the network changes. Data mining is the science of extracting implicit, previously unknown, and potentially useful information from large data sets or databases. Network fault management based on data mining not only has advantage in subject areas, but also has a significant meaning. This dissertation utilizes data mining in the alarm correlation analysis, and then studies the alarm association rules mining and alarm predictive patterns mining in specific communication environment including the dynamics of the network topology and business, the different alarm attributes, the uniformity of network business priority level, as well as the rare number of alarms used to predict major network faults. The main research results and innovations can be included as follows:(1) Novel preprocessing methods of alarm data are proposed. In order to convert original alarms to the suitable data for mining, double constraint based sliding window method is first proposed. After deleting redundancy data, quantifying data and extracting data, the alarm data finally can be converted to the alarm transactions. Meanwhile, according to the unequal attributes of alarms and characters of the actual communication networks, two-neuron neural network is designed to confirm the alarm weights. The weights of the neural network may reflect the knowledge of the experts, or change automatically when the inputs change. Finally, a preprocessing expert system which includes alarm transactions extraction and the alarm weights determined is provided, for it can deal with the original alarms more scientifically and efficiently.(2) Advanced approaches and innovative methods on alarm association rules mining are proposed. Firstly, a WPFP-tree structure is provided to mine weighted frequent patterns. Based on the WPFP-tree, two weighted association rules mining algorithms WPFPT-WARM and WPFPT-WARM* are proposed respectively. The main idea of WPFPT-WARM is to generate weighted frequent patterns by the candidate itemsets gradually. In contrast, WPFPT-WARM* is first on the basis of WPFP-tree to find the maximum length weighted frequent patterns, and then generate all the weighted frequent patterns gradually. Experimental results show that the WPFP-tree based weighted association rules mining methods have higher efficiency and lower complexity compared to the current methods. Totally speaking, WPFPT-WARM is more suitable for mining short alarm patterns, and the complexity of this method is relatively low. On the other hand, WPFPT-WARM* can handle the long alarm patterns with higher efficiency.(3) Based on communication alarm data, advanced approachs on incremental updating mining are proposed. The author presents two incremental updating mining methods, in which WPFPT-WARM(S) is used when the minimum weighted support threshold changes and WPFPT-WARM(D) is used when the database changes. Both algorithms are based on the mining algorithm WPFPT-WARM. Experimental results show that the proposed two incremental updating mining algorithms have higher efficiency in comparison with other updating methods, because it adopts the method to update WPFP-tree structure directly.(4) The author considers the problems of generating and dealing with the rules. The author first considers the depth search based rule generation algorithm, and proposes an advanced method DFS-RG'; Due to the attributes inequality of alarms, a weighted frequent pattern based rule generation method WFP-RG is proposed; Considering some rules are redundant and some have no relationship, the author find a new method to deal with these problems. Tests on the actual alarm database prove that the methods to generate rules and deal with rules are correct and effective.(5) The author proposes a Sparse Bayesian based Method APPM-SBL to learn the rare alarm data and find predictive patterns. APPM-SBL uses the sparse Bayesian linear classifier to learn the sample data and generate predictive patterns. It proves that APPM-SBL can not only avoid over-study like SVM, but also has higher performance and fewer nuclear functions than SVM.(6) To verify and evaluate the proposed algorithms in this dissertation, alarm correlation analysis system and weighted association rules mining platform are developed. Based on this platform, the performances of all proposed algorithms are evaluated with actual alarm database in communication networks.