Kernel-Based Deep Learning

Many theoretical and practical problems in science and engineering can be summarized as learning objective functions from a limited number of sampled data.The acquisition of sampled data is usually accompanied by noise.In order to reduce the impact of noise on subsequent data processing methods,it is natural to require a stable sampling process.Therefore,the reproducing kernel Hilbert space(RKHS)has become an ideal background space for processing point valued data.Machine learning methods based on reproducing kernel Hilbert space have become a research hotspot in the field of machine learning.Through the composite operation of functions,deep neural networks exhibit strong expressive power in function representation,thereby improving the approximation ability of functions.This characteristic has made the deep learning method a great success.Combining the advantages of reproducing kernel method and deep learning method,Bohn et al.proposed a deep learning method based on reproducing kernel.This method uses the composition of functions in RKHS to represent the objective function,and then establishes a learning method.This paper analyzes the deep learning method based on reproducing kernel and applies it to function reconstruction problems.Firstly,using the representation theorem established by Bohn et al.,we transform the interpolation problem and regularization problem in infinite dimensional space into a nonlinear optimization problem with finite coefficients,and use the quasi Newton method to establish a solution algorithm for the transformed optimization problem.Secondly,the established learning method is analyzed and its disadvantage is pointed out,that is,it usually requires a large amount of computation when solving nonlinear optimization problems.In order to solve this problem,we apply the idea of parameter sharing to the method,thereby reducing the amount of computation caused by computational gradients.Finally,we apply the developed method to the function reconstruction problem.Through numerical experiments,the reconstruction effects of reproducing kernel based learning methods,reproducing kernel based deep learning methods,and parameter sharing methods are compared.