Design And Implementation Of Cyber Kill Chain Detection System Based On Machine Learning

With the persistent improvement of social information,network security incidents are more and more frequent,resulting in incalculable economic losses and social resources waste.At the same time,the innovation of hacker technology and cyber attack mode also makes the traditional network defense technology cannot meet the needs of this era.Cyber Kill Chain is a new type of attack model which belongs to Advanced Persistent Threat.It is used by more and more hackers for information stealing and system attack because of its multi-stage nature,and the diversity of technical tools makes the intrusion behaviors very hidden on the target host or target network.Therefore,how to effectively and completely mine and predict the Cyber Kill Chain is a new challenge in the field of network security.Compared with the traditional method based on rule-matching,the network flow logs can monitor the user's access behavior and abnormal events of the system,which is the main way to effectively analyze the Cyber Kill Chain with the characteristics of concealment,persistence and multi-stage.Therefore,a Cyber Kill Chain Detection and Prediction Model is studied and proposed in this thesis.And a Cyber Kill Chain Detection System is designed and implemented by analyzing and mining the network logs.The specific work of this thesis is as follows: 1.The characteristics and network logs of Cyber Kill Chain are analyzed,and the Cyber Kill Chain Detection and Prediction Model is proposed.Firstly,constructs the feature vector is constructed and the similarity measure of the field is defined.With regard to the problem of subjectively constructing the feature vector,the Unsupervised Feature Selection based on Local Graph Reconstruction algorithm is proposed.The analysis indicates that the algorithm considers the problem of parameter-free and redundancy at the same time,obtains the global optimal solution by solving the matrix quadratic form,and then selects the m-dimensional features that can best keep the original data spatial structure.Secondly,to solve the problem that the traditional spectral clustering algorithm needs to set the number of clusters,an Improved Spectral Clustering with Automatic Cluster-Number Identification algorithm is proposed.By sparse regularizing the indicator vector of clusters,the position of non-zero value contained in a indicator vector corresponds to the data points of this cluster.Experimental results show that the proposed algorithm can automatically identify the potential clusters number,i.e.each cluster represents a Cyber Kill Chain attack sequence.Finally,because the Cyber Kill Chain sequence set has been obtained,three kinds of Cyber Kill Chain variant models is proposed and Markov model is first applied for scenario construction and theoretical analysis,and the attack prediction results of Cyber Kill Chain is deduced by solving Kolmogorov differential equation.2.Based on the Cyber Kill Chain detection and prediction model,with the help of big data platform,web framework,database theory and other technics,the Cyber Kill Chain detection system is designed and implemented.There are four modules in the system,which are data preprocessing module,Cyber Kill Chain detection and prediction module,visualization module and retrieval module.The data preprocessing module completes the normalization of the original data and the calculation and storage of the similarity between the samples.Then the redundant features are removed by the feature selection algorithm.Cyber Kill Chain detection and prediction module mine the Cyber Kill Chain sequence through improved clustering algorithm.The detected kill chain sequence set is stored and used as the input of the Cyber Kill Chain prediction.Then,according to the theoretical analysis,the possibility of future attacks is outputted.The visualization module adopts the descriptive visual analysis method to realize the multi-level and efficient expression of data.The retrieval module completes the multi condition retrieval of user input and gives detailed information.