Distribution Preserving-based Feature Extraction Methods

With the rapid development of society,pattern recognition has received much attention in all the fileds of areas and has become a practical application in the current world.The research content of pattern recognition includes many disciplines such as mathematics,machine learning,computer vision,artificial intelligence,neuroscience,and cognitive science.It is a challenging theoretical research and a rapidly developing challenging application.As the core research of pattern recognition,feature extraction mainly studies how to learn the discriminant attributes that are useful for recognition tasks from the high dimensional observation data.The quality of feature extraction model directly determines the performance of pattern recognition.The research of feature extraction has become one of the interesting issues in the field of pattern recognition area.The existing models obtain the optimal low dimensional representations of data according to the Euclidean distances,and without considering the nonlinear manifold structure hidden in the high dimensional data.Moreover,the Euclidean distance based models cannot guarantee that the manifold structure with a large intrinsic curvature is mapped into the eigen-dimensional embedding space.In this thesis,the kernel density estimation method is used to approximate the eigen-dimensional manifold structure hidden in high dimensional data space.Several distribution preserving embedding based nonlinear feature extraction models are proposed to eliminate the large intrinsic curvature of data.And we verify the performance of these proposed feature extraction models from the perspective of supervised classification,unsupervised clustering,semi-supervised classification.The main content and contribution of the thesis can be summarized as follows:(1)An edge smoothing-based distribution preserving hyperspherical embedding(DPHE)methed is proposed for hyperspectral image classification and the methed can project high dimensional hyperspectral data into a low dimensional hyperspherical coordinate system.Specifically,when we estimate the distribution of each pixel point via the spectral feature,the spatial information and the intensity information of the hyperspectral data are fully utilized.And the distribution,that is smoothed by the edge stop function,can detect the truthful edge information of the real object.Therefore,the proposed methed is able to capture the intrinsic geometry structure embedded in the hyperspectral data and then keep the structure in the lower dimensional hyperspherical embedding space as much as possible.The experimental results of the three hyperspectral datasets also show that the DPHE can detect the intrinsic geometry structure of hyperspectral data and significantly improve the supervised recognition results.(2)A distribution preserving-based deep semi-nonnegative matrix factorization(DPDSNMF)method is proposed.From the perspective of clustering,using the deep semi-nonnegative matrix factorization technique,the method can obtain the hidden hierarchical representations according to the unknown attributes of the given data.On the other hand,the intrinsic geometry structures of each cluster can be described by the distribution of data within the cluster.In the DPDSNMF method,the kernel density estimation method is used to approximate the manifold structure embedded in the data,and then the hierarchical representation will respect to the data manifold structure by explicitly maintaining the consistency of the two distributions.The method can completely preserve the intrinsic geometry structure embedded in the original high dimensional data space in the discriminant projection space.And the unsupervised clustering results also verify the effectiveness of the model.We design an adaptive method to efficiently optimize the constraint objective function of the proposed method.(3)We proposed a distribution preserving network embedding(DPNE)method,which uses a non-negative constraint autoencoder to learn a low dimensional part-based embedding.In the DPNE method,we use the neighborhood kernel density estimation to reveal the truthful manifold structure embedded in the high dimensional data space.And then we seek the deep part-based embedding,that will respect to the intrinsic manifold structure.The goal of preserving the manifold structure in the low dimensional space is achieved by introduced a regularization of distribution consistency.In addition,compared to other traditional and deep embedding methods,the unsupervised clustering results on the image and text datasets also show that the proposed method can better preserve the intrinsic manifold structure hidden in the high dimensional data space in the low dimensional embedding space.(4)Distribution preserving semi-supervised deep embedding(DPSDE)method is proposed to solve the issue,that the Euclidean distance-based method in the semi-supervised learning framework cannot capture the manifold structure between a small amount of labeled samples and a large amount of unlabeled samples.Because the data distribution can approximate the truthful manifold structure embedded in the high dimensional data space,and the within-class samples are located in the continuous high density region,and the between-class samples are connected through some low density regions.Combining the geometric information of a small number of labeled samples and a large number of unlabeled samples,the proposed method can not only learn an effective classification decision surface,but also learn a low dimensional embedding that can preserve the intrinsic geometry.In addition,the semi-supervised recognition results on the image datasets also show that the proposed method can learn an effective classification decision surface.