Research On Indefinite Kernel Based On Regression Algorithms

Kernel method has been widely used in machine learning fields such as classification,regression and clustering because of its good generalization performance.In the regression problem,the support vector regression with kernel function is an effective method to solve the problem of non-linear regression.Due to the limitation of traditional statistical learning theory,most existing support vector regression algorithms require the introduction of positive definite kernels satisfying Mercer's theorem.However,on the one hand,the kernel function can not guarantee the semi-positive definite of the kernel matrix in real application scenarios,which leads to the existing positive definite accounting method no longer applicable;on the other hand,the learning performance of indefinite kernel function often achieves better than the positive definite kernel's.Therefore,it is of great significance to introduce indefinite kernels into support vector regression and solve the regression problem effectively.Indefinite Kernel Support Vector Regression(IKSVR)maps the original features into the reproducing kernel Krejan space,which transforms the optimization problem of support vector regression into a non-convex problem.The existing indefinite kernel methods have some problems,such as missing some important information in the data or duality gap,which affect the fitting performance and generalization performance of the model.To solve the above problems,a single indefinite kernel support vector regression algorithm with convex difference programming is proposed in this paper.Furthermore,considering that multi-kernel learning can effectively improve the efficiency of selecting kernel functions and their parameters,and can improve the fitting performance of support vector regression algorithm to a certain extent when dealing with complex data.Therefore,based on the IKSVR algorithm,this paper proposes a multi-indefinite kernel support vector regression algorithm.The main research work of this paper is as follows:1)A single indefinite kernel support vector regression algorithm IKSVR is proposed.Firstly,the optimization process of the original problem model and dual problem model of support vector regression is analyzed,and the relationship between the two solutions is studied to determine the existence of dual gap.Secondly,convex difference programming is introduced,and convex difference algorithm is used to optimize the original indefinite kernel support vector regression problem.IKSVR algorithm is proposed,and it is proved that the algorithm can converge to the local optimal point.Finally,the effectiveness and convergence of the algorithm are verified by experiments.2)A multiple indefinite kernel support vector regression algorithm MIKSVR is proposed.A multiple indefinite kernel support vector regression model is constructed by linear combination of different or multiple kernels.The multiple kernel model is optimized by using alternating optimization strategy and IKSVR algorithm,and the convergence of the algorithm is analyzed.The experimental results show that the MIKSVR algorithm converges and the fitting performance are better than other comparison algorithms.