Three Kinds Of Efficient Kernel Machines

The success of Support Vector Machines is owed to the combination of statistical learning theory and kernel theory, which at the same time springs the appearance of kernel machines and its rapid development. Kernel machines have become a new research direction in the field of machine learning. Research in this thesis is focus on the following three kinds of kernel machines:(1) Hidden space kernel machines. At first, utilizing least squares loss function inthe hidden space to measure empirical risk, we propose Least Squares Hidden Space Support Vector Machines (LS-HSSVMs). Following that, Sparse Hidden Space Support Vector Machines (SHSSVMs) is presented to overcome the shortage of having no sparseness for LS-HSSVMs.(2) Wavelet kernel machines. Based on the good approximation property of orthogonal wavelet scale function, we directly use it to construct kernel function, and propose an algorithm named Wavelet Kernel Based Ridge Regression (WKRR). WKRR does not require the kernel function satisfying the Mercer's positive definite condition, and moreover, the construction of kernel is simple and easy to be realized.(3) Bayesian kernel machines. Based on a rank-1 update, we propose Sparse Bayesian Learning Algorithm (SBLA), which has low complexity and high sparseness, thus being very suitable for large-scale problems.Theory analysis and simulation results show the validity and feasibility of these algorithms.