Application Research Of Spark-based Moth-flame Optimization Algorithm

Nowadays,the Internet has slowly entered everyone's life from the national level to the enterprise level.The Internet has become another new revolution in the 21 st century,but it also faces many problems such as the processing of massive data and information security.In the era of big data,the rapid growth of operators,the competition and develo pment of major network ecommerce platforms,how to process data in real time,and ho w to ensure information security.How to reduce the dimension of data to increase the pr ocessing rate of data has also been a hot issue in our research.In this context,the traditional intrusion detection technology can not achieve the desired effect,and intrusion detection technology has begun to develop in the direction of artificial intelligence and distributed.In addition,the traditional serial computing mode can no longer meet the processing of massive data,The distributed platform of cluster mode has gradually replaced the serial computing mode,especially the computing platform of big data.The memory-based Spark distributed computing framework has unique advantages.In this thesis,by studying the advantages and disadvantages of the traditional intelligent optimization algorithm-Moth-Flame Optimization(MFO)algorithm,then optimizes and improves it,and implements the parallel operation on Spark distributed computing framework.The main content of this thesis is as follows.Introduce the traditional Moth-Flame Optimization(MFO)algorithm and analyze the deficiencies,then make binary improvements to the algorithm,and use the binary MFO algorithm for feature selection.Due to the massive data cannot be effectively processed in the stand-alone mode,the traditional MFO algorithm is parallelized with Spark.The traditional MFO algorithm and the improved MFO algorithm were tested on the Spark cluster and stand-alone mode,respectively,and compared in terms of acceleration ratio,accuracy,and effectiveness.Finally,it was verified that the Spark-based MFO algorithm has good performance and be able to process the massive data that needs to be iterated more efficiently and stably.In view of the problems that MFO algorithm is prone to fall into local optimality,the classification accuracy is not high,and the global convergence speed is poor.In this thesis,Cauchy jumps are used to initialize the population,and the convergence factor of the MFO algorithm is improved by introducing an exponential function,and chaos perturbation is performed on the current optimal moth individual.And an improved Moth-Flame Optimization algorithm is proposed to avoid premature convergence and local optimization.The improved MFO algorithm effectively improves the classification accuracy of traditional MFO algorithm and avoids premature phenomenon.Preprocess massive high-dimensional data,and then reduce the dimension of the data.In this thesis,three dimension reduction methods are used to reduce the dimension of network intrusion detection data to reduce the number of features to be detected.Support vector machine is used to classify the data,distinguish the abnormal data,and distinguish the intrusion detection types.Experiments show that this method can effectively reduce the dimensionality of data,reduce the operation efficiency of the algorithm,and improve the classification performance of the algorithm and intrusion detection efficiency.