Kernel Based Learning Machines

In the early of 1960s the theory of machine learning based on the databegan to be studied. It had not been well developed until Vanpik et al completed Statistical Learning Theory (SLT) and proposed a new general and efficient machine learning algorithm.Support Vector Machines (SVMs) in the 1990s. The concept of capacity control of a learning machine plays an important role in the SLT. It is necessary for a learning machine to achieve a good balance between theempirical risk and capacity control of it to obtain a good generalization performance. Before SLT was presented and accomplished, kernel functions had been introduced into machines learning. In other words, the techniques of nonlinear mapping and nonlinear function had been used in machine learning. However the first example of thesuccessful applications of kernel functions in machine learning is the SVMs. That is because the introduction of nonlinear functions into machines leaning makes the set ofhypothesis function without capacity control become very wide, thus the overfitting problem and the decrease the generalization occursineluctably. It is the combination of SLT and kernel method that spring the appearance of kernel machine and its rapid development. At present, the main kernel machine includes SVM, kernel Fisher classifier, kernel principal component analysis (PCA), etc. From the viewpoint of the combination of SLT and kernel method, this dissertation studied the analysis and the improvement of SVMs and new kernel machines. In the aspect of the analysis and the improvement of SVMs, four parts work are studiedas follows.1.Some geometry ofSVMs for classification and regression is described and proven. And then the generalization performance of SVMs on new-added samples is discussed. Through the analysis of the property of new-added samples and the effect of them on support vectors and non-support vectors, some valuable results are presented. These enable us to conclude that SVM has a good compatibility, adaptability and generalization performance for new-added samples and is a hereditable learning model.2.Based on the analysis of the conclusions in the statistical learning theory, especially the VC dimension of linear functions, linear programming SVMs are presented. In linear programming SVMs, the bound of the VC dimension is loosened 西安电子科技大学博士学位论文目录Vproperly. Simulation resultsfor both artificial and real data show the generalization performance of our method is a good approximation of SVMs and the computation complex is largely reduced by our method.3.An unconstrained convex quadratic programming for support vector regression (SVR) is proposed, in which Gaussian loss function is adopted.Due to no linear constraint some classical multi-dimensions optimization algorithms such as steepest descend method, Newton method, conjugate gradation method and so on can be used to solve the convex quadratic programming for SVR.Compared with standard SVR, this method has a fast training speed and can be generalized into the complex-valued field directly. Experimental results confirm the feasibility and the validity of our method.4.Generally the wireless channel varies with time. Therefore it is necessary for a multi-user detection algorithm to have acapacity ofadaptation. Adaptive SVMs are presented for multi-user detection.Structural Risk introduced in SVMs leads to the more generalization and less training samples to be required than the other learning models and the nonlinear SVMs can approximate optimum multi-user detector when the adaptive SVMs is used for multi-user detection. In the other aspect of new kernel machines based on SLT, Four algorithms are described. 1.A new constructiveprinciple, which depends on the distribution of examples, for measuring the generalization performance is proposed based on the analysis of the generalization performance of SVMs.Our principle is consistencyin geometry with that in statistical learning theory, composed of two-order statistic of samples and shows the convergence...