Structural Risk Minimization Principle On Rough Space

Statistical Learning Theory is important regarded as a sound framework that handles a variety of learning problems in presence of small size data samples. However, Statistical Learning Theory is built on probability space and based on real random samples, so it can hardly handle statistical learning problems built on rough space and based on rough samples, which can be encountered in real world scenarios. Structural risk minimization principle is one of the kernels content of Statistical Learning Theory. The principle is the theoretical fundamentals of establishing the support vector machine. Based on above, the structural risk minimization principle based on rough samples is explored in the paper. Firstly, the concepts of annealed entropy, growth function and VC dimension as well as their properties on rough space are given. Secondly, the constructive and distribution-independent bounds with VC dimension on rough space are presented. Thirdly, the structural risk minimization principle on rough space is proposed. The consistency of this principle is proven, and asymptotic bounds on the rate of convergence is derived. Finally, the structural risk minimization principle's direct implementation on rough space is given.