Research On Vulnerability Detection Method Based On Big Data Analysis

With the popularity of mobile terminals,the data traffic of the network is increasing rapidly and the information transmission rate is faster,the network security vulnerability becomes more and more prominent.The traditional methods of vulnerability detection cannot satisfy the needs of people on efficiency in the big data era.Combining with big data analysis techniques to detect vulnerability exploited has practical significance to enhance the security of the network.this thesis proposes a detection scheme for the vulnerability exploitation,this scheme includes the big data security analysis platform based on Spark and the network security equipment processing module.The big data security analysis platform to analysis the real-time data flow of network,extract attack features in the exploit process,and using the ability of big data analysis that efficient handle mass data to improve the detect efficiency of the exploit.The platform integrates the Flume collection,Kafka and Spark-streaming calculation module,extract a part of the scan and DoS attack feature,using the cart decision tree algorithms,constructing attack prediction model,realize detecting the exploit,then,sent the abnormal data flow and normal data flow to the corresponding network security equipment processing by network security equipment processing module,to prevent vulnerability have been successfully exploited.Through the experiment to verify the good performance of big data security analysis platform.Network security equipment processing module include network security equipment routing scheme,in order to adapt to the environment with network security function virtualization deployment,we proposes a security device routing scheme based on SDN network,which uses SDN technology that separate the data plane and control plane of the network can flexible control the network traffic.the scheme combined the improved shortest routing algorithm of inline and passive security devices,and the shortest-path routing algorithm using neural network,get a highly efficient routing policy of security devices.In this thesis,the security data for big data security analysis platform is still relatively simple,the type of vulnerability can be detected is not comprehensive,only found the remote vulnerability exploited.Later need to collect more information about the vulnerability exploited,optimize the classification algorithm,so that the detection method can detect more.In order to solve the problem of network congestion,it is necessary to take into account it for security device routing scheme.