Random Mapping Approach To Model Selection Of Large-Scale Kernel Methods

Model selection is the bottleneck of and the key to theoretical research and practical application of large-scale kernel methods.Most of the current large-scale kernel methods select the kernel functions and set the model parameters by the rule of thumb in reproducing kernel Hilbert spaces(RKHSs),which lacks not only solid theoretical guarantees but also computationally efficient methods for model selection.To address these issues,we propose the random mapping approach to model selection of large-scale kernel methods,which maps the original problems into explicit random hypothesis spaces rather than RKHSs and can obtain consistent results with those of the classical model selection in RKHSs under statistical guarantees.The main results are as follows:1.We propose the circulant random feature mappings.Through approximating shiftinvariant kernel functions using harmonic analysis and circulant random matrix projection,we first propose the signed circulant random feature mapping(SCRF).Then under the Boosting framework,we construct the data-dependent circulant random feature mapping using our proposed SCRF.Furthermore,we investigate the effectiveness and efficiency of our proposed methods through analyzing their unbiasedness,variance and computational complexity.2.We propose the novel approach to model selection of large-scale kernel methods with explicit random hypothesis spaces.Using our proposed circulant random feature mappings,we construct the circulant random hypothesis space and the heterogeneous random hypothesis space explicitly.Further,we study the feasibility and effectiveness of our alternative approach with respect to the original one via RKHSs through analyzing the convergence of model selection,the generalization errors of the selected models,and the computational complexity in our constructed random hypothesis spaces.3.We propose the general scheme for model selection of large-scale kernel methods.We first map the original model selection of kernel methods into explicit random hypothesis spaces,in which we design and implement efficient learning algorithms for selecting kernel models.Further,we propose the general scheme for model selection of large-scale kernel methods with linear computational complexity.In summary,we not only propose the state-of-the-art random feature mapping methods but also present a statistically unbiased approach with random hypothesis spaces and a general efficient scheme for model selection of large-scale kernel methods,which lays a solid foundation for the theoretical development and practical application of model selection of large-scale kernel methods.