Sparse Learning Machines And Their Applications In Hyperspectral Imagery Classification

In the real applications,most of the available machine learning technologies main face the problem that it is unable for the users to learn the model from given data automatically,thus the learning model should be pre-customized.Determining the appropriate model requires the users to have a wealth of domain knowledge and pay a great amount of effort,which limits the application of machine learning technology.In order to solve this problem,this thesis proposed to start with a large network structure,and then prune the network one-step by data-driven,so as to obtain a learning machine composed of a small number of network nodes,and its generalization performance will not reduce.The specific work is as follows:(1)The sparse learning machine is theoretically analyzed.First,the definition of sparse learning machine is presented.Second,by utilizing the learning theory and compressed sensing,it is proved that by only using a few coupled compressed observations,a sparse learning machine without performance loss can be successfully obtained with a very large probability and its sparse level is no higher than a quantity determined by the number of measurements and the coherence between the sensing matrix and the representing matrix.Furthermore,by taking the sparse learning machine with Hilbert space regularization as an example,the specific generalization error bound of sparse learning machine is analyzed.(2)A data-driven coupled compressive pruning method is proposed for Least Square Support Vector Machine(LS-S VM),thus a discriminative sparse learning machine model is obtained and applied to the classical pattern recognition problems such as regression and binary clas-sification.Among the support vector machines,LS-SVM is computationally attractive for reducing a set of inequality constraints to linear equations.In this thesis,inspired by the recently developed compressive sampling theory,a one-step compressive pruning strategy is proposed to construct a sparse LS-SVM without the remarkable reduction of accuracy.It is a fast,universal and information-preserved pruning approach that can avoid the intensive computations in iterative retraining in most of the available pruning methods.(3)It is proposed to train the LS-SVM and Laplacian Support Vector Machine(LapSVM)by coupled compressive sensing,thus a supervised and semi-supervised discriminative sparse learning machine model is obtained and applied to the hyperspectral imagery classification problem respectively.In the LS-SVM and LapSVM models,all the training samples are involved as the support vectors,which loses the sparseness of SVM model and increases the cost of computation and storage.To overcome this difficulty,this thesis proposes a training method denoted as coupled compressive sensing for the LS-SVM and LapSVM by data-driven and obtain the corresponding sparse learning machine models.Without pruning,this training method can improve the sparseness of these two models in the training process and there is no loss of generalization performance.(4)By introducing the tensor representation into the sparse coding based classifier,which is a special kind of sparse learning machine,two tensor sparse coding based classifiers are proposed and applied to the hyperspectral imagery classification problem.First,to make full use of the spatial information of hyperspectral imagery,each certain hyperspectral pixel and its spatial neighbors are represented in the form of spatial neighbor tensor.And the tensor sparse coding is implemented on this tensor representation form to obtain a sparse learning machine model denoted as Sparse Coding based Classifier with Spatial Neighbor Tensor(SCC-SNT).Owing to utilizing the spatial structural information of hyperspectral imagery sufficiently,SCC-SNT based hyperspectral imagery classification can get accurate classification results with only a small number of training pixels.In addition,to limit the uncertainty of labeling the mixed pixels,a regularization term which maximizes the likeli-hood of sparse coding coefficients is introduced into SCC-SNT.Thus this sparse learning machine model could still obtain satisfactory results when it is applied to predict the label of mixed pixel.