Improvement And Application Of The Parallel SVM Algorithm Based On GPU

With the development of science and technology, the high-performance computing has been a hot area of research. The GPU (Graphic Processing Unit) is an integral part of high-performance computing. GPU was initially used in the field like 3D graphics processing and video codec, however, as it has strong power of floating-point calculation, parallel computation and high memory bandwidth, it has developed to GPGPU (General Purpose GPU) which is highly parallel and programmable and has been used in scientific computation. In data mining, the GPU technology is also used. SVM (Support Vector Machine) is a classic data mining algorithm which has long execution time and large memory usage. The long execution time of SVM lies in the processes of training model and parameter optimization. The training process aims at building a model. The existing training algorithms all need a number of iterations, what’s more, they need to repetitively compute the kernel matrix and access to the memory, so the modeling process is very slow especially when the data size is large and the data dimensionality is high. The parameter optimization process aims at getting the best classification results. The current optimization algorithms all need a number of iterations and the complex training process is called in all the iterations, which will lead to a boom in calculation amount. Therefore, how to speed up the SVM process to make it be fitter for the applications with much bigger data size and much higher need of real-time, has wide researching prospect.Based on the study of SVM training algorithms and parameter optimization algorithms, this paper first improves the parallel training algorithm based on GPU, then proposes the parallel parameter optimization algorithms and last applies the new SVM algorithm to the intrusion detection. The main work includes:(1) This paper elaborates on the fundamentals of SMO (Sequential Minimal Optimization), one of the training algorithms, describes the basic procedure of SMO, and analyzes the principle and parallel mechanism of P2SMO (Parallel-Parallel SMO) algorithm which is the parallel version of SMO algorithm based on GPU. Considering the characteristic of the parallel mechanism, this paper finds the disadvantages of P2SMO using theoretical analysis and experimental analysis. Then, this paper improves P2SMO by introducing the condition of quick convergence and changing the searching strategy. Experimental data show that the improved P2SMO algorithm can not only guarantee the classification accuracy, but also increase the classification speed.(2) SVM parameter-optimization algorithms have long execution time, as they have to call the complex training process again and again. It is necessary to parallelize the optimization process. Based on the improved P2SMO algorithm, this paper describes how to use the parameter optimization algorithms of grid search and particle swarm optimization in SVM. By combining the improved P2SMO with grid search and particle swarm optimization, the paper first proposes parallel versions of these two algorithms to optimize the parameters in SVM. Experiments indicate that the new parallel algorithms can both obtain almost same optimization results as original ones, at the same time, the optimization efficiency are increased greatly. Moreover, the performance of parallel particle swarm optimization is better than that of grid search.(3) This paper combines the improved P2SMO and the parallel particle swarm optimization to SVM modeling process, which is then applied to intrusion detection. Comparing the experimental results of this new method with those of LibSVM, results show that the new method can not only guarantee the intrusion detection accuracy, but also sharply reduce the execution time.