Kernel Based Learning Machines  Posted on:20040719  Degree:Doctor  Type:Dissertation  Country:China  Candidate:W D Zhou  Full Text:PDF  GTID:1118360122960278  Subject:Circuits and Systems  Abstract/Summary:  PDF Full Text Request  In the early of 1960s the theory of machine learning based on the databegan to be studied. It had not been well developed until Vanpik et al completed Statistical Learning Theory (SLT) and proposed a new general and efficient machine learning algorithm.Support Vector Machines (SVMs) in the 1990s. The concept of capacity control of a learning machine plays an important role in the SLT. It is necessary for a learning machine to achieve a good balance between theempirical risk and capacity control of it to obtain a good generalization performance. Before SLT was presented and accomplished, kernel functions had been introduced into machines learning. In other words, the techniques of nonlinear mapping and nonlinear function had been used in machine learning. However the first example of thesuccessful applications of kernel functions in machine learning is the SVMs. That is because the introduction of nonlinear functions into machines leaning makes the set ofhypothesis function without capacity control become very wide, thus the overfitting problem and the decrease the generalization occursineluctably. It is the combination of SLT and kernel method that spring the appearance of kernel machine and its rapid development. At present, the main kernel machine includes SVM, kernel Fisher classifier, kernel principal component analysis (PCA), etc. From the viewpoint of the combination of SLT and kernel method, this dissertation studied the analysis and the improvement of SVMs and new kernel machines. In the aspect of the analysis and the improvement of SVMs, four parts work are studiedas follows.1.Some geometry ofSVMs for classification and regression is described and proven. And then the generalization performance of SVMs on newadded samples is discussed. Through the analysis of the property of newadded samples and the effect of them on support vectors and nonsupport vectors, some valuable results are presented. These enable us to conclude that SVM has a good compatibility, adaptability and generalization performance for newadded samples and is a hereditable learning model.2.Based on the analysis of the conclusions in the statistical learning theory, especially the VC dimension of linear functions, linear programming SVMs are presented. In linear programming SVMs, the bound of the VC dimension is loosened è¥¿å®‰ç”µåç§‘æŠ€å¤§å¦åšå£«å¦ä½è®ºæ–‡ç›®å½•Vproperly. Simulation resultsfor both artificial and real data show the generalization performance of our method is a good approximation of SVMs and the computation complex is largely reduced by our method.3.An unconstrained convex quadratic programming for support vector regression (SVR) is proposed, in which Gaussian loss function is adopted.Due to no linear constraint some classical multidimensions optimization algorithms such as steepest descend method, Newton method, conjugate gradation method and so on can be used to solve the convex quadratic programming for SVR.Compared with standard SVR, this method has a fast training speed and can be generalized into the complexvalued field directly. Experimental results confirm the feasibility and the validity of our method.4.Generally the wireless channel varies with time. Therefore it is necessary for a multiuser detection algorithm to have acapacity ofadaptation. Adaptive SVMs are presented for multiuser detection.Structural Risk introduced in SVMs leads to the more generalization and less training samples to be required than the other learning models and the nonlinear SVMs can approximate optimum multiuser detector when the adaptive SVMs is used for multiuser detection. In the other aspect of new kernel machines based on SLT, Four algorithms are described. 1.A new constructiveprinciple, which depends on the distribution of examples, for measuring the generalization performance is proposed based on the analysis of the generalization performance of SVMs.Our principle is consistencyin geometry with that in statistical learning theory, composed of twoorder statistic of samples and shows the convergence...
 Keywords/Search Tags:  Statistical learning theory, Kernel machine, Support vector machine, Reproducing kernel, Reproducing kernel Hilbert space, Pattern recognition, Regression estimation, Mathematics programming, Complexvalued sample, Principal component analysis  PDF Full Text Request  Related items 
 
