Research On Intrusion Detection Based On Spark+ Technique

With the development of science and technology,the Internet plays an irreplaceable role in People's Daily life.However,due to the opening and sharing characteristics of the Internet,various network intrusions are increasingly frequent.As a global Internet user,China's cyber invasion,commercial information leakage and other behaviors had become an important obstacle to the development of China's information industry and the building of an information society.Therefore,how to effectively manage the security log information,analyze and study the intrusion detection behavior,so as to protect personal privacy and trade secrets,is an urgent problem in the field of information security in China.As an effective way to prevent network attack behavior,intrusion detection technology is one of the research hotspots of researchers at home and abroad.Most researchers tend to combine machine learning algorithms with intrusion detection technology to establish intrusion detection systems.However,traditional intrusion detection algorithms detect the false positive rate is high and the single-machine processing speed is slow,which can't effectively deal with the unknown attack behavior.In view of this,an improved intrusion detection system based on Spark+CUDA platform is proposed to improve the Adaboost algorithm.The intrusion detection model detects the false positive rate and improves the operational efficiency of the intrusion detection system.The main research contents are as follows:(1)Firstly,the research status of the intrusion detection field and the current research problems are analyzed.Then the attack type of the intrusion detection and the specific classification method of the intrusion detection system are introduced in detail.(2)Secondly,Aiming at the problems of high false positive rate and long processing time of traditional intrusion detection system,an improved Adaboost algorithm is proposed.Firstly,the algorithm is improved from two aspects: sample point weight and weak classifier weight,and samples are added in the sample point weight improvement.Judging the correct rate of N rounds before the point,it is more accurate to increase the sample weight value of the classification error and reduce the sample weight value of the classification when the sample point is iteratively updated.In the weak classifier improvement,the difficult sample points are classified according to the specific weak classifier.The advantage is that the weak classifier weight is appropriately adjusted when the classifier is combined,and the weight distribution of the weak classifier in the combined classifier is further optimized by the sample point correct rate parameter control.On the other hand,Experiments in the KDD CUP 99 dataset show that the anomaly detection accuracy of the improved algorithm is higher than that of the traditional algorithm and the false positive rate is lower than the traditional algorithm.(3)Finally,due to the improved Adaboost algorithm needs to compare the correct rate and error rate of the sample points in the iterative process,the running efficiency of the improved algorithm is greatly reduced.In view of this,the improved Adaboost algorithm is parallelized on the Spark+CUDA platform.The CUDA platform will improve the calculation of sample point weights in the Adaboost algorithm.The N-round correctness rate T(i)and the false-error rate F(i)calculation before the sample point,the sample point error rate calculation,and the weak classifier weight calculation are complicated.It is executed by the GPU.On the other hand,the Spark cluster is used to block the data set.Then Experiments with KDD CUP 99 dataset and self-acquired web log dataset show that this method effectively improves the operational efficiency of the improved Adaboost algorithm.