Research On Model Selection Problem For Twin Support Vector Machines

Twin Support Vector Machines(TWSVM) is a machine learning method which is proposed on the basis of Proximal Support Vector Machines(PSVM) and Proximal Support Vector Machine based on the generalized eigenvalues(GEPSVM). Compared with PSVM、GEPSVM, TWSVM solves a hyperplane for each type of sample points.The two hyperplanes in TWSVM have no constraint on the parallel condition. The binary classification problem is converted to two smaller quadratic programming problems by TWSVM. Thereby the training time is reduced to 1/4 of SVM, it can solve the XOR problem better, and the classification performance is better than PSVM and GEPSVM. But, as the same as support vector machines, TWSVM still need to solve the model selection problem. The performance of TWSVM depends on the selection of penalty parameters, kernel function and parameters in the kernel function, and how to choose the kernel function and parameters is the model selection problem. In this paper, we research TWSVM starting with the model selection problem to improve its performance. The following are the main research contents of this paper:Firstly,this paper studies the feasibility and significance of using Quantum Particle Swarm Optimization to optimize parameters in TWSVM and then proposes Twin Support Vector Machines based on Quantum Particle Swarm Optimization(QPSO-TWSVM). QPSO-TWSVM search the optimal parameters by the use of the global searching ability of the Quantum Particle Swarm Optimization(QPSO), and avoid itself using the empirical values to specify the parameters, which is too blind. Experiments show that QPSO-TWSVM improves the classification accuracy of TWSVM.Then,this paper studies the feasibility and significance of using the Mixed Kernel function to solve the problem of selecting the kernel function in TWSVM. The Mixed Kernel function uses a global kernel function and a local kernel function to construct a new mixed kernel function which has the better performance.And then this paper proposes Twin Support Vector Machines based on the Mixed Kernel function(MK-TWSVM). To further solve the problem of parameter selection in MK-TWSVM, this paper further studies the feasibility of using the shuffled frog leaping algorithm to optimize parameters in MK-TWSVM. On this basis,this paper proposes Mixed Kernel Twin Support Vector Machines based on the shuffled frog leaping algorithm(SFLA-MK-TWSVM). Experiments show that SFLA-MK-TWSVM improves the classification accuracy of TWSVM.Finally, this paper studies the advantages of wavelet analysis technology and the feasibility of applying the kernel function constructed based on wavelet analysis to TWSVM. Wavelet analysis has the characteristics of multivariate interpolation and sparse change, and the wavelet kernel function based on wavelet analysis can approximate any nonlinear functions, it is also suitable for the analysis of local signals and the detection of transient signals. On this basis,this paper proposes Wavelet Twin Support Vector Machines(WTWSVM). Experiments show that the wavelet kernel function has improved the classification accuracy and generalization ability of TWSVM greatly, and it also expands the range of selecting the kernel function in TWSVM. But, WTWSVM also faces the same problem that the parameters are difficult to be determined.So this paper further studies the Glowworm Swarm Optimization to optimize parameters in WTWSVM and proposes Wavelet Twin Support Vector Machines based on Glowworm Swarm Optimization(GSO-WTWSVM).