Efficient Ensemble-based Neighborhood Preserving Strategies For Dimension Reduction And Classification

Dimension reduction(DR)and data classification are two most important machine learning tasks,used for many real-world pattern recognition applications such as face recognition,medical imaging,fingerprint identification,speech recognition etc.Neighborhood preserving strategies are developed in many well-known algorithms,such as neighborhood preserving embedding(NPE),local preserving projection(LPP)and k-nearest neighbor classifier(KNN).However,these algorithms are very sensitive to such settings;for example,NPE and LPP are very sensitive to neighborhood parameter,which decreases the performance of dimension reduction.Moreover,existing manifold DR methods generally make use of a single graph for preserving neighborhood relations i.e.not suitable for multi-view datasets due to lack of thorough discrimination.Also,classification performance of KNN is highly influenced by the neighborhood size k and existing outliers.In this thesis,efficient ensemble-based neighborhood preserving strategies are devised aimed at diminishing the aforementioned constraints in NPE,LPP and KNN.In the first approach,we proposed a novel DR method called weighted neighborhood preserving ensemble embedding(WNPEE).Unlike NPE,the proposed WNPEE constructs an ensemble of adjacent graphs with the number of nearest neighbors varying.With this graph ensemble building,WNPEE can obtain the low dimensional projections with optimal embedded graph pursuing in a joint optimization way.Experiments on ORL,Georgia Tech,CMU PIE and Yale four face databases demonstrate that WNPEE achieves competitive recognition rate than NPE and other comparative DR methods.Also the proposed WNPEE achieves much lower sensitivity to neighborhood parameter and able to preserve more local manifold structure of high-dimensional data when compared to NPE and other related DR algorithms.Furthermore,a second DR approach called an ensemble graph-based locality preserving projections(EGLPP)is proposed.EGLPP extends the ensemble framework of WNPEE to enhance the dimensionality reduction performance of LPP.EGLPP constructs a homogeneous ensemble of adjacency graphs and finally uses the integrated embedded graph to optimize the low-dimensional projections.Moreover,being motivated by the performances of WNPEE and EGLPP which uses an ensemble graph embedding framework,we proposed a generalized multi-manifold graph ensemble embedding framework(MLGEE)for multi-view datasets.MLGEE leverages an ensemble regularization term to consider the multi-manifolds information from the heterogeneous graphs to consider the intrinsic geometrical structure of the multi-view data distribution.Extensive experiments on four face recognition datasets for EGLPP and six multiview datasets of handwritten numerals recognition,object recognition and face recognition on MLGEE demonstrate their comparative robustness.Finally,a local mean based k-harmonic nearest centroid neighbor classifier(LMKHNCN)is proposed to enhance the KNN classification performance.LMKHNCN considers the distancebased proximity as well as spatial distribution of k-nearest neighbors.In LMKHNCN,initially the k-nearest centroid neighbors in each class are found which are used to find k different local mean vectors,and then employed to compute their harmonic mean distance to the query sample.Then,the query sample is assigned to the class with minimum harmonic mean distance.The experimental results based on twenty-six real-world datasets shows that the proposed LMKHNCN classifier achieves lower error rates,particularly in small sample-size situations.Also,LMKHNCN shows very less sensitivity to parameter k when compared to the related four KNN-based classifiers.