The Research And Application Of Real-time Data Processing Technology In Network Security Management System

With modern communications technology continues to evolve and progress,the network has been rapid popularization and promotion,it has become the people's indispensable part of life and work.In the context of the booming network,network security incidents occur frequently,it attracted wide attention.Faced with this situation,network security management personnel used lots of security devices in the network,and managed uniformly by using network security management system.However,network security devices of different functions and duties generate a lot of information,which is not only difficult to use but also has caused tremendous pressure to network security management system.Therefore,the timely processing of vast amounts of information has become the development trend of network security management system.In order to process and analyze a large number of real-time information,this paper will present Storm,which is widely used in real-time data processing,into the network security management system and will redesign the event management based on Storm,trying to improve the efficiency of processing real-time data by Storm.The main work and results of this paper are as follows:1.This paper divided the process into 5 sub-processes,including event collection,event standardization,event preprocess,advanced event process and event display.In this paper,the event management module has been improved.Based on Storm,this paper designed and implemented each of the five sub-processes of event management,combining the actual demand and streaming data processing to improve the efficiency of the system.Compared by Java program on single PC,the Storm-based program has significantly improved the speed of processing data.2.Faced with the problem of redundant related events in the Storm-based design of event preprocess,this paper used the mechanism of allowing cache in Storm's Bolt and used the method of feature-based classification to discover the relation in events,merging the events which have the same feature and removing the redundant data effectively.3.According to the characteristics of Storm's streaming data process,this paper has transformed the process of event merging in advanced event process,adding the work of counting merging events to event display to ensure the streaming data process.Because the program frequently queried the event rules in the event rule matching process,this paper stored the event rules in Redis,improving the efficiency of rule querying.