Research And Implementation On Large Scale Network Data Stream Anomaly Detection System

With the rapid development of the Internet, which brings us a lot of convenience, we face menaces from all kinds of security incidents. There are more and more attacks to net resource, such as DOS, worm etc. It's an urgent problem to solve that how to check abnormal network data in order to provide information to incident response person.The characteristic of the large-scale network data stream is continuous, rapid, large, while the availability of today's intrusion detection system dealing with is limit, and there is no data stream mining technology to solve practical network data detection and analysis problem. This paper designs large-scale network data stream anomaly detection system model and network data stream frequent pattern mining and detection arithmetic. We realized the system, and made relative practice. The designs are theoretic and practical.The large scale network data stream anomaly detection system model is made up of four sub-models, which are data collection and pre-disposal model, frequent pattern mining model, anomaly detection model, data analysis and response mode. In data collection and pre-disposal model, slide time window technology is used to collect network data. The first N site ports and their relative information are taken and scattered, and save them in the summary database. In frequent pattern mining model, CLOSET arithmetic is used to mine frequent patterns of network data stream, and save them in the frequent pattern database. In anomaly detection model, the nearer and more frequent patterns are matched previously, and powered Euclid distances is used to account the similarity, and the anomaly data is sent to data analysis and response model. The data analysis and response model, managers detect the data based on the fact. According to the real anomaly detection, managers make the incident response process, while feedback the...