Research On Feature Learning Algorithm For High-dimensional Data

The processing of high-dimensional data has been a hot issue in the field of machine learning.Because processing high-dimensional data directly will face " Curse of Dimensionality " and "Algorithm Failure",many effictive feature learning methods have been proposed for solving these problems,including principal component analysis,linear discriminant analysis,etc..These methods,however,have various problems in complex,highly nonlinear and multi feature scenarios,how to make full use of the original features Information,reduction and fusion of high-dimensional features is still a very challenging problem.To solve these problems,this dissertation designs a scientific and effective feature learning method to reduce the dimension of data and retain the effective features of data.The main work is divided into the following aspects:1 This dissertation extends the traditional LDA and KDA algorithms,and proposes a kernel discriminant learing analysis(KDLOR)method of ordinal regression.This method uses the data information in a global way.Considering the distribution information of different classes,it uses the sequence information of classes to carry out the ordered regression,which overcomes the defects of neglecting the global information and high computational complexity existing in the existing ordered regression algorithm.On this basis,combined with the idea of orthogonal projection vector,a feature extraction scheme based on the combination of multiple projection vectors is proposed.The scheme is divided into two stages:the first stage is to recursively obtain the projection vector through orthogonal space,and search for the optimal projection vector in the orthogonal subspace of the obtained projection vector;the second stage uses different combination strategies to combine the decision rules of the projection vector,so as to form the final decision-making mode,which enables the algorithm to extract more original information.Compared with the classical algorithm in the ordinal regression experiment,the algorithm has very good performance.2 In order to solve the problem which is difficult to obtain satisfactory performance with a single feature,a feature fusion method based on manifold learning is proposed,which integrates multiple features into a single embedding direction.This method is to find a low dimensional subspace after dimension reduction and redundant and irrelevant information can be eliminated.This method is extended to solve two differnent machine learning problems:supervised learning and semi supervised learning.The experimental results show that the proposed method is better than other feature fusion methods in classification performance and algorithm stability.3 A feature fusion method based on combinational kernel is proposed.This method uses multiple high-dimensional features at the same time,and uses different kernel functions to extract the corresponding features respectively.The SVM classifer is used for finnal classification and only one classifer is needed,which greatly reduce the complexity of calculation.Compared to traditional methods that multiple classifiers are needed,the proposed method can increase the computational complexictiy.We also provide two different combination methods for final decision results:simple average method and weighted average method.Experimental results show that the proposed feature fusion method outperform singl classification methods.