Research On Two-phase Learning Algorithm Based On Kre(?)n Space

The two-phase learning algorithm is a learning framework that divides some traditional machine learning algorithms from the original one-phase problem solving process into two different learning phases,while the essence of problem solving remains the same.Initially,the two-phase learning algorithm was introduced to solve the illconditioned problem faced by kernel Fisher discriminant analysis(KFDA)when processing high-dimensional small sample data.Specifically,the two-phase kernel Fisher discriminant analysis(TP-KFDA)algorithm is the essence of the KFDA problem solution,it transforms the KFDA problem into an equivalent solution that kernel principal component analysis(KPCA)and Fisher discriminant analysis(FDA)were solved in turn.However,the TP-KFDA algorithm still has two limitations:(1)the choice of kernel functions is not abundant;(2)the application problem is single.On the one hand,the TP-KFDA algorithm requires that the kernel function must meet the constraints of positive definiteness,that is,the kernel function used must satisfy Mercer's theorem,which limits the rich selection of kernel functions to a certain extent.The latest research shows that in some cases,the application of indefinite kernel can often better describe the relationship between data,and in many practical applications,it can obtain better modeling capabilities than positive indefinite kernel.On the other hand,the TP-KFDA algorithm can only be used to solve the dimensionality reduction problem of singlemodal data,and is not applicable to other problems,such as multi-modal data analysis problems,classification problems,etc.,which will seriously limit the practical application of the TP-KFDA algorithm.Therefore,this article will start from these two aspects to study and improve the TP-KFDA algorithm.The research results are summarized as follows:1)In view of the problem that the existing TP-KFDA algorithm is limited to the use of positive definite kernels,this paper proposes a two-phase indefinite kernel Fisher discriminant analysis(TP-IKFDA)algorithm based on Kre(?)n space,which introduces the indefinite kernel into the TP-KFDA algorithm and improves the choice of kernel function flexibility.Specifically,the TP-IKFDA algorithm first performs non-linear dimensionality reduction preprocessing on the original data sample through the first phase of indefinite kernel principal component analysis(IKPCA),and then performs Fisher discrimination analysis in the feature space after dimensionality reduction analysis by IKPCA.In order to further explain the rationality of the TP-IKFDA algorithm,based on the definition of the IKFDA problem in Kre(?)n space,the theoretical derivation proves that the solution of the IKFDA problem is essentially an FDA problem in the space after IKPCA dimensionality reduction.Finally,classification experiments are performed on some high-dimensional small sample data sets,the experimental results show that the classification performance of the TP-IKFDA algorithm is better than the existing IKFDA algorithm.2)As the TP-KFDA algorithm is only used to solve the dimensionality reduction problem of single-modal data,and for the problems that are difficult to use for multimodal data analysis,this paper proposes a two-phase indefinite kernel canonical correlation(TP-IKCCA)algorithm for solving multi-modal data problems.Specifically,the TP-IKCCA algorithm first performs non-linear dimensionality reduction preprocessing on the original data through IKPCA in the first phase,and then performs canonical correlation analysis(CCA)in the second phase in the feature space after dimensionality reduction by IKPCA.In addition,based on the definition of indefinite kernel canonical correlation analysis(IKCCA)problem in Kre(?)n space,this paper proves that the essence of IKCCA problem is the CCA problem in the feature space after IKPCA dimensionality reduction.Finally,experimental results on real data sets show that the generalization performance of the TP-IKCCA algorithm is significantly improved compared to the existing IKCCA algorithm.3)As both TP-IKFDA and TP-IKCCA are used for the purpose of data dimensionality reduction,the scope of application still has certain limitations.This paper further proposes a two-phase indefinite kernel support vector machine(TPIKSVM)algorithm.The TP-IKSVM algorithm divides the solution process of indefinite kernel support vector machine(IKSVM)problem into IKPCA and support vector machine(SVM)two phases,in order to make full use of IKPCA's advantages in processing high-dimensional data,and finally combine the supervision information to effectively construct the classification model SVM.In addition,this paper re-defines the IKSVM problem based on Kre(?)n space,and proves that the essence of the IKSVM problem is the SVM problem in the feature space after dimensionality reduction by IKPCA.Finally,experimental results on real data sets show that the classification accuracy of the TP-IKSVM algorithm is better than the current mainstream IKSVM algorithm.