A Study On Feature Selection Algorithm Based On SVM-RFE And Particle Swarm Optimization

With the emergence of large-scale and high dimension data in data mining and machine learning fields, feature selection is becoming one of the most important problem in Pattern Recognition. Too many irrelevant and redundant features will not only cause the curse of dimensionality, but impact the performance of classifiers directly, so feature selection is important. Feature selection is removing irrelevant and redundant features from the original feature space according to a certain evaluation criteria, to achieve the goal of reducing the feature dimension. Support Vector Machine (SVM) is an effective recognition tool that being widely used in many application areas, and has unique advantages in small sample or high-dimensional data. In this paper, Support Vector Machine is used as classifier for feature selection.Based on different evaluation criteria, many different and effective feature selection methods have been proposed, but for complex problems, feature selection needs to constantly improved. In this paper a two stage feature selection method based on SVM-RFE and Particle Swarm Optimization (PSO) algorithm is proposed. SVM-RFE belongs to the heuristic sequence backward selection algorithm with fast speed, but can’t recognize the redundant features effectively. PSO algorithm is a kind of intelligent optimization algorithm with random search. This method can find the optimal solution, but the result has high uncertainty. In order to reduce data dimension we’ll use the advantages of both SVM-RFE and PSO. Firstly, it ranks features by the SVM-RFE algorithm and removes some irrelevant features to reduce data dimension preliminary. Then the better subset of SVM-RFE initializes the initial PSO population with good starting points.So SVM-RFE can provide experience and reduce search space for PSO algorithm behind, and improve the efficiency of selection to some extent. In addition, using the algorithm of adaptive weighting parameters to avoid premature convergence and other issues. In the end, we verified the effectiveness of the method through different datasets form UCI database. The experience results demonstrate that our approach can get smaller size subset under the condition of without reducing classification performance, and in high-dimensional data sets, the algorithm running time is less than the particle swarm optimization.