Research On Indefinite Kernel Support Vector Machine Algorithms

Kernel Methods have become an effective tool in machine learning because of its good expressive ability and nonlinear mapping.The most representative application of the kernel methods is support vector machine(SVM).Constrained by the development of traditional sta-tistical learning theory and optimization theory,most of the SVM methods require kernel func-tions to satisfy the positive definite condition.On the one hand,this constraint restricts the rich expression of the kernel function,which is limited to the representation in the Hilbert space;on the other hand,in many practical problems,the use of the indefinite kernel can better describe the relationship between the data,so as to achieve better performance than the positive defi-nite kernel.Therefore,the introduction of indefinite kernel is of great importance for further research of kernel methods.Indefinite kernel support vector machine(IKSVM)has become a non-convex problem as the introduction of indefinite kernel and the optimization of this problem is a research hotspot in machine learning nowadays.Most of the existing methods are designed for the dual problem of IKSVM,but because the primal and dual problems of IKSVM arc not convex,the solution of the dual problem is not equals to the primal problem's,which further affects the performance of classifiers.In order to solve this problem,we start with the primal IKSVM problem and pro-pose a single indefinite kernel support vector machine algorithm based on difference of convex functions programming(IKSVM-DC).Furthermore,considering that multiple kernel learning(MKL)can effectively avoid the selection of kernel parameters and improve the performance of classifiers,we propose a learning algorithm of multiple indefinite kernel SVM based on dif-ference of convex functions programming(MIKSVM-DC).Aiming at the problem of indefinite kernel support vector machine,two main works are done in this paper:1)A new algorithm IKSVM-DC based on difference of convex functions programming(DCP)for single indefinite kernel learning is proposed.First of all,we introduce the DCP into the primal IKSVM model and propose an algorithm named IKSVM-DC.Then,we prove that the proposed algorithm can converge to a local optimum according to the theoretical analysis of IKSVM-DC.Furthermore,we extend the proposed algorithm to multi-class classification problems and propose an unified indefinite kernel model to solve the multi-class IKSVM problem.Finally,the effectiveness of the proposed algorithm is verified by experiments.2)A multiple indefinite kernel learning algorithm MIKSVM-DC based on DCP is proposed.Combined with the IKSVM-DC algorithm,a two step iterative optimization algorithm is used to solve the MIKSVM problem,and the convergence analysis of the proposed algo-rithm is carried out in this paper.Finally,the experimental comparison between the proposed algorithm and the existing MKL methods shows that the proposed algorithm has better per-formance.