Feature Selection Methods Based On Particle Swarm Optimiza- Tion

Machine learning method is faced with the problem of increasingly high dimen-sional data. The performance of commonly used learning algorithms will decrease dramatically on data with lots of features. Feature selection, as a solution to this prob-lem, has always been paid attention to. Particle swarm optimization is an optimization method with good local search capability. Since a good search method is always needed in feature selection methods, researchers are showing great interest in applying particle swarm optimization in feature selection problem. Based on in-depth analysis of both particle swarm optimization and feature selection problem, we mainly focus on two categories of research work. The first category is to improve the performance of parti-cle swarm optimization itself, and the second category is to make the feature selection problem and particle swarm optimization combinative.First, we analyse standard binary particle swarm optimization, and propose a new particle swarm optimization method based on average fitness and fitness proportional selection. On the contrary to using distance information to calculate the probability of updating a particle’s position information, we propose to adopt fitness proportional selection based on average fitness of a set of particles that have the same value on a par-ticular dimension of their position components. We carry out a series of experiments on function optimization problem and multi-dimensional knapsack problem. Experimen-tal results show that the method we proposed has better search capability comparing with the standard binary particle swarm optimization.Second, we propose a feature selection method based on domain knowledge and particle swarm optimization. This method provides a way to take advantage of the domain knowledge of feature selection problem to guide the search process of particles so that the search process of particles will be more efficient. We carry out experiments on 10 data-sets to test the convergence speed and the ability to find good optima of our method. Experimental results show that with domain knowledge involved, the new method can speed up particles’convergence.Finally, based on the two methods mentioned above, we propose a fast feature se-lection method based on particle swarm optimization. The new method takes advantage of both domain knowledge of feature selection problem and average fitness to judge a feature. We aim to improve the search efficiency of particles on feature selection prob-lem. In addition to the 10 data-sets mentioned above, we conduct more experiments on two big data-sets with 500 and 10000 features respectively. The experiments produce the expected results...