Approximate Model Selection For Kernel Methods Via Matrix Analysis

Model selection of kernel methods is fundamental to the theory and application of kernel methods.Previous model selection criteria are usually difficult to compute or have high computational complexity.The kernel matrix contains all values of the kernel function on all data points.Therefore,this thesis studies approximate model selection for kernel methods via matrix analysis.The main results are as follows.1.To develop the theory of approximate model selection,we define the notion of approximate consistency to study under what conditions,and at what rate,the approximate model selection criterion converges to the accurate one.Under the risk-minimization criteria,we analyze the approximate consistency of Nystr（?）m method and multilevel circulant matrix（MCM）approximation.2.We propose approximate model selection criteria via MCM.We define a model selection criterion by minimizing the generalization error bound about the covering number of the ball of RKHS,propose an approximate model selection criterion via MCM,and prove the approximate consistency between the approximate and accurate criteria.We construct a novel type of MCM from the data and the kernel function,define an approximate model selection criterion via the eigensystem of the MCM,and prove the approximate consistency between the approximate criterion and the criterion defined by the mean discrepancy of the positive and negative classes in the feature space.3.We propose approximate automatic kernel selection approaches by utilizing the combination of multiple multilevel circulant matrices respectively for the L₁ regularization and L₂ regularization of the combination weights.We prove the consistency between the approximate hypothesis produced with multilevel circulant matrices and the optimal hypothesis in the hypothesis space.In summary,on the basis of matrix analysis,we develop the theory of approximate model selection,and propose efficient and theoretically sound approximate model selec-tion criteria and approaches.