Design And Implementation Of Distributed Anomaly Detection And Analysis System Based On Network Traffic

The Internet industry has entered a period of rapid development,which has brought people a lot of unprecedented experience in recent years.The rich application scenes bring much convenience to people,but it also exposes a number of network security problems.The more diverse of the Network traffic types are,the more difficult of the network security monitoring is.This actuality makes the quality of network communication and the security of users’ hosts face the threat of network intrusion at all times.Therefore,it is of great value and significance to study how to efficiently and accurately discover the abnormal network data from massive network traffic data in real time,prevent network security risks and provide early warning services for network security maintenance personnel.This paper studies the current network traffic anomaly detection method.In view of the limitations of the data scale and processing ability in the previous methods that have problems of low accuracy and real-time monitoring difficulties.Combined with machine learning and big data technology,a streaming parallel anomaly detection method based on multi-model fusion is proposed.Firstly,we use the features of network traffic to train and generate the multiple single models,then fuses them together by Stacking,so it can meet the need of improving the detection accuracy and then make the model parallelized to make it possible to have a distributed processing about massive streaming data on a basis of ensuring algorithm accuracy.Finally,with the abnormal data expanded continuously and then extract the key features,so that it can form an abnormal network traffic blacklist then use the blacklist to accurately match real-time network traffic data and quickly detect abnormal data.The accuracy and effectiveness of this method are verified by using the KDD CUP99 data set.Through a large number of contrast experiments,it is proved that compared with other typical anomaly detection algorithms,the method given in this paper can batch flow data in real time,and improve the accuracy and efficiency of anomaly detection.In order to make an intuitive display analysis of network traffic data and abnormal detection results,based on the application of the anomaly detection method given in this paper,a network traffic anomaly detection visualization system based on Spring Boot and Vue technology is also developed.The system realizes the storage and query of network traffic,analyzes and displays the data with the help of Echarts technology,and finally realizes the visual display of the abnormal detection results of traffic data.