The Theoretical Study Of Kernel Methods And Its Application In Pattern Recognition

The kernel method and its theory are based on a bijective function and transformation theory,and its core is to research some positive definite functions,its related theory and applications in Hilbert space.The kernel-based machine learning method is not only applicable to the pattern expressed by eigenvectors,but also the pattern expressed by structural datasets.The former corresponds to the vector kernel method,and the latter corresponds to the graph kernel method.Therefore,the kernel method of pattern recognition can be divided into two categories:vector kernel and graph kernel.It mainly focuses on the study of vector kernel early.Vector kernel has been successfully developed in theory and its applications.Many scholars have applied and developed the theory and application technology of machine learning based on vector kernel method.In recent years,graph kernel is gradually familiar to us,and it is applied and developed.Especially,in structural image modeling,feature description and matching of digital image,it is paid attention to by more and more scholars.Graph kernel may describe the structural characteristics of the graph,so it has a particular advantage in structural pattern recognition.Kernel method has its solid theoretical foundation.Its theory had been developed in mathematical theory,pattern recognition,machine learning,data mining and other related fields.Therefore,it is of great significance to further study the kernel method,its theory and applications.The contributions of this thesis are as follows:(1)A multi-kernel learning method with reproducing property is proposed.Firstly,the fundamental solution of a kind of generalized differential equation is introduced by Dirac function,and its fundamental solution is analyzed and discussed.Secondly,a multi-kernel learning method based on 2-space reproducing kernel is designed.This multi-kernel can enhance the interpretability of decision function of support vector machine(SVM)and improve the classification performance of SVM.Finally,experiments are carried out to verify the effectiveness of our method.(2)A novel multi-attribute convolution kernel method with reproducing property is proposed.Firstly,we introduce the solution of a kind of general differential equation by Dirac function,and then design a multi-attribute convolution kernel function based on this solution.Secondly,we verified and obtained that the multi-attribute function satisfies the condition of Mercer kernel,and this multi-attribute kernel function has three attributes:L1-norm,L2-norm and Laplacian kernel.Thirdly,compared with traditional kernel method in Hilbert space,the convolution kernel method may integrate the advantages of each attribute and improve the classification abilities of SVM based on multi-attribute kernel function.Finally,by experiments,we verified the effectiveness of our method.(3)A hybrid graph kernel method based on Weisfeiler-Lehman(WL)graph kernel is proposed.Firstly,the basic theory and related knowledge of the WL graph kernel family are introduced.Based on the sequence of the WL graph,the subtree kernel,the edge kernel and the shortest path kernel are further given.Secondly,three hybrid graph kernels are defined based on the WL graph kernels.The first one is the weighted combined graph kernel(WCGK),which is a parameter kernel.The second one is the combined graph kernel based on the accuracy ratio(ARCGK).The third one is the product combined graph kernel(PCGK).The latter two graph kernels belong to nonparametric kernel.Finally,the experiment results show that the hybrid graph kernel based on WL graph kernels can comparable with or overcome the corresponding single graph kernel in our experiments,so it is very important to expand the theory and application of hybrid graph kernel.(4)A reproducing graph kernel method based on approximation von Neumann entropy and reproducing kernel is proposed.Firstly,we introduce an approximated expression of information entropy for an undirected graph,which depends on the vertices of the graph,and then measures the structure information by this approximated von Neumann entropy.Secondly,we introduce the H1-kernel function in H1-space by the fundamental solution of a general differential equation.Finally,we define an approximated von Neumann entropy reproducing graph kernel(AVNERGK)based on the approximation von Neumann information entropy and the H1-kernel function.The experimental results show that,compared with other graph kernel methods,the classification accuracy of our method can comparable with or overcome the selected advanced graph kernels.In the time complexity,computation time of our method is shorter than other graph kernel methods.