Research And Design On Distributed Agent Framework For Log And Data Collection

Computer network is widely applied in various fields and network security has been become more and more concerned by institutions and enterprises. While firewalls, anti-virus systems, IDS, vulnerability scanning systems etc. have been deployed in the network, lots of safety equipments are the lack of effective integration. Facing mass of log information generated by the equipments, network managers are often difficult to deal with them, and network security threats remain prominent. In order to combine the original isolated safety equipments to form a unified whole and respond to a variety of security events consistently, we propose research and development of Unified Network Security Management Platform (UNSMP) that is a comprehensive security solution.Data collection agents of UNSMP collect and analyze logs and alarms from all kinds of safety equipments, and the control center of UNSMP analyzes comprehensively all the alarms and assesses the whole risk of the managed network, which ensures safety events and corresponding responses to be dealt with timely. Collecting logs efficiently and flexibly from all kinds of safety equipments is an important task at unified network security management. However, due to extensive sources of logs, different formats of logs and amount of log records, there are considerably difficulties in collecting logs. Therefore, this paper presents a distributed agent framework for data collection that can be used to collect logs in a variety of different structures, different sizes of networks.This paper, based on the overall analysis to UNSMP, studies the plug-in technology of data collection and the types of events, and designs and realizes Data Collection Agent Model (DCAM) with agent technology. On the basis of DCAM, further studies and designs the distributed agent framework for data collection and it' s mechanism of multi -level fusion for events and the communication between components. The framework can be applied to different sizes of networks with several aspects of flexibility and extensibility, powerful real-time monitoring capability, fusion of multi-level mechanism for events, secure and cost-effective data communication.