Design And Implementation Of A AAA Service Monitoring System Based On Log Analysis

With the increasing of ISP's network business types(2G\3G\4G, WIFI, LAN and so on) and rapid growth of users, the requirements of AAA systems' performance and function become higher and higher. To full fill the requirements, the type and amount of composition equipment of AAA system increases at the meanwhile, which leads to more and more software/hardware failures and malicious attacks to different equipment. Because of impact and interdependence between various devices, single device failure may lead to several types of failures logs which are generated by multiple devices and the structures of log files are different. All of above made it more and more difficult to confirm the scope of failure, locate the source of fault or attack by analyzing the log data.To solve these problems above, we completed the following four tasks in this paper:Firstly, an Auto Log Collection and Template Extraction(ALCTE) mechanism is proposed. The log data auto collecting function is based on flume and all the logs are converted to text format. All the words in log files are divided into Temple Word and Data Word based on their occurrence frequency in logs. And by do this, a log item is decomposed into a log template and a data vector and the format of different types of logs are normalized. ALCTE is used to solve the auto analysis difficult which led by the device type, software version, network level differences.Secondly, based on formative log files, we design an event based clustering of log data method(Co LDFFE). By Co LDFFE, the correlation between fault event(such as database down) and log items is detected. The log items which are related to one fault event are assembled to fault detecting, locating and analyzing.Thirdly, in view of the deficiencies of the existing anomaly detection policy in performance and effectiveness, an Attack Source Detection Mechanism Based on TF-IDF(ASDBT) is proposed. Base on the recent one year's AAA server log files of a telecommunications company, the proportion of authentication results and the main anomaly types are discussed at first. By calculating the correlation between the data source to be screened and the abnormal data source set, the other abnormal data sources are soon determined.Lastly, Base on the recent one year's AAA server log files of a telecommunications company, by integrated use of ALCTE, Co LDFFE and ASDBT which is proposed above, a prototype system of AAA service status monitoring is designed and implemented. And the simulations based on server different scenarios(such as link down, database down and malicious login attack) shows that our prototype system is quite effective and efficient.