Research On Network Security Situational Awareness Based On Flow Association Mining

With the advent of the Internet of everything era,social production and life are increasingly dependent on the Internet,but at the same time,more and more lawbreakers also use the Internet to launch network attacks on individuals and enterprises,threatening the security of property and privacy.Therefore,with the development and progress of science and technology,network security gradually occupies an extremely important position in the security field.Network security situational awareness technology is a cognitive process of the security state of network system.It is a method to analyze the security of current network environment from a large amount of network data,and can effectively find the potential threats in the network.However,with the arrival of the era of big data,the data traffic in the network has shown an exponential growth.In addition,the cyber attacks have become more covert and diversified.A network security situational awareness system suitable for the huge network flow environment has become an urgent need for individuals,enterprises and even the country.Although in recent years,more and more experts and scholars have applied cutting-edge technologies to the research of network security situational awareness,there are still many problems.First,most optimization methods for data processing only choose more suitable machine learning algorithms,without considering the impact of feature redundancy on the performance of the overall model.Second,situation data is mostly acquired by simulating network attack,and no feasible data acquisition scheme is proposed in high-speed data flow environment.Thirdly,most of the existing researches are the optimization of theoretical models and algorithms,and no detailed design scheme of network security situational awareness system has been proposed.Aiming at the above problems,this thesis studies the network security situational awareness technology based on flow correlation mining,and realizes the design and deployment of the prototype system.Aiming at the effect of feature redundancy on the performance of the model,a flow feature association mining algorithm based on spearman coefficient is proposed in this thesis.According to spearman coefficient,this algorithm analyzes the correlation among traffic characteristics,then combines the meaning and value range of features,fuses or removes the features with strong correlation,and recodes them,thus improving the performance of the machine learning model.The experimental results show that after the stream feature association mining algorithm processes the data,and the feature redundancy is removed.The accuracy,F1 value and AUC value of the logistic regression model are improved.Next,in the situation assessment part,this thesis divides the features into shallow features and deep features.The shallow features that can be directly extracted from the network packet combined with the prior knowledge of attack behaviors can pre-warn the attack behaviors and improve the real-time performance of the system.The deep features obtained through accumulation and statistics use flow feature association mining algorithm to remove feature redundancy according to the correlation between features,ensuring the balance of system accuracy and real-time performance.Finally,based on the algorithm proposed in this thesis,a prototype network security situational awareness system is designed and deployed.The system is mainly divided into data acquisition and transmission module,system resource monitoring module,shallow layer feature warning module and deep layer feature correlation mining module.In terms of the implementation scheme and selection of each module,several commonly used technologies and frameworks are compared,and the optimal scheme is selected based on the overall requirements of the system.The usability of each module of the prototype system has been tested,and all functions can run normally.