Application Research Of Spark Based Grey Wolf Optimization Algorithm

In the 21 st century,both big data and Internet finance have made great progress.The importance of the state in this field has gradually increased.Peer-to-peer(P2P)is an innovative lending model that is a powerful complement to the traditional financial industry.Compared with the traditional financial lending industry,it is more efficient and convenient.However,with the rapid development of technology and finance,it has also brought quite serious challenges to this industry,such as user defaults and other risks.To reduce risks for the industry through technological means,but with the rapid development of technology,the data volume of this industry has shown an explosive growth trend.The huge data volume and massive characteristics appear to be weak in the traditional computing field,but the current big data computing And artificial intelligence just fills this gap.In this paper,by studying the advantages and disadvantages of the traditional intelligent optimization algorithm-Gray Wolf Optimization Algorithm(GWO),the algorithm is optimized and improved,and combined with the big data computing platform Spark to implement the algorithm.The main contents of this article are as follows:(1)Introduce and analyze the shortcomings of the traditional gray wolf optimization algorithm,and combine the actual experimental scenarios to improve the gray wolf optimization algorithm in binary.Due to the huge amount of data,the algorithm cannot effectively process the data amount when the algorithm is running locally.The traditional algorithm is parallelized,and the traditional GWO algorithm and the improved GWO algorithm are tested in the local environment and the distributed environment.Compared with the convergence and speed of the algorithm,the final experimental results show that the improved gray wolf optimization algorithm based on Spark can process a large amount of data more quickly,efficiently and steadily.(2)For the gray wolf optimization algorithm,it is easy to fall into local optimization and slow convergence speed in iterative calculation.In this paper,the Gaussian distribution is used to map the randomly initialized population;in the gray wolf population position update,the Gaussian-Cauchy combination mutation operator is used.For perturbation,this paper proposes an improved binary gray wolf optimization algorithm.Experimental results show that the improved CBGWO effectively improves the convergence speed problem and premature phenomenon of traditional GWO.(3)For a large number of high-dimensional credit data sets,data preprocessing is performed first,then feature selection is combined with CBGWO,and important features are selected as much as possible,and finally combined with classifier modeling.In this paper,logistic regression and random forest classifier are used for modeling and model tuning respectively.The final experimental results show that the improved graywolf optimization algorithm based on Spark combined with random forest predicts the model with the best effect,and at the same time improves the computing performance of the model.