Hierarchical Manifold Learning With Applications To Supervised Classification

The very high resolution(VHR) remote sensing satellite data, such as Quick Bird and IKONOS, have been widely used in many fields. In recent years, the earth observation has entered a new era of high resolution as the continuous and successful launch of “GF-1” and “GF-2” satellites in our country. The requirement for the VHR remote sensing image processing capabilities is also increasing with the increasing application demands. However, currently there is relatively lower level of automation in VHR remote sensing images processing. Therefore, it has important application value to solve some key technical issues of automatic classification for VHR remote sensing images and full play their effectiveness.The object-oriented image classification has been one of the most important VHR remote sensing image processing methods. In order to make good use of the fine-textured information of VHR remote sensing images, various kinds of features, such as spectra, shape, texture features, are extracted to represent the image objects. But, this can cause high-dimensional features in object-oriented classification, thus the curse-of-dimensionality happens. Dimension reduction is a feasible way to address the aforementioned challenges. Therefore, in the object-oriented supervised classification of VHR remote sensing images, dimension reduction is a key technical issue that need to be solved in this doctoral dissertation.Over the past 10 years, dimension reduction has been received more and more attentions in the fields of machine learning and pattern recognition. Those linear methods fail to discover the true intrinsic underlying structure of the high-dimensional data which lying on the nonlinear manifold. As one of the most representative nonlinear dimension reduction methods, manifold learning has made many successful applications in the fields of information processing. However, when it is applied to supervised classification, in particular,hierarchical classification,the result is still unsatisfactory. To address this issue, hierarchical manifold learning(HML) is proposed in this doctoral dissertation. In the dimension reduction process we extract sharing features to represent parent-manifold’s information, and better solve the out-of-sample problem of manifold learning by using generalized regression neural network(GRNN) at considerably lower computational cost. The propose HML is applied to supervised classification of VHR remote sensing images. Compared with other manifold learning methods, the proposed HML can achieve better classification performance.The major contributions and innovations from this dissertation are summarized as follows:(1) Currently most manifold learning methods assume that all the high-dimensional data points lie on a single-layer manifold, and the multiple manifolds learning lacks the hierarchical characteristic. It is very difficult for the existing manifold learning methods to tackle multi-level classification. To this end, this paper gives the concepts of “submanifold” and “parent-manifold”. On this basis, we propose the HML algorithm, which takes into account both the local geometric structural information within class and the class-label information between classes simultaneously. The hierarchical manifold is constructed based on the given training sets in a bottom-up way,and the optimal nonlinear mapping function is calculated and then applied to dimension reduction. Hence, the final embeddings in the low-dimensional subspace are more sufficient for classification.(2) Most manifold learning algorithms are unsupervised learning methods. When those algorithms are applied to supervised classification, they will encounter the out-of-sample problem. A simple yet effective solution to tackle the out-of-sample problem can not be found so far. In this paper the GRNN is proposed to sovle the out-of-sample problem. Two important parameters in GRNN, i.e., the smoothing factor and the threshold value, have a great influence on the performance of GRNN.(3) In the process of building the hierarchical structure of our proposed HML, we will encounter another problem, i.e., how to represent parent-manifold in bottom-up way based on the subclass samples features. We propose the concept of sharing feature to depict parent-manifold. The existing method of sharing features extraction is based on Ada Boost,it has shown many disadvantages. A new method of sharing feature extraction based on the feature mapping technique is proposed to extract sharing features.The essence of the new method is to compute the generalized Rayleigh. The sharing features obtained in this way can be used to represent and construct parent-manifold.(4) Based on the above-mentioned methods, two typical data, i.e., the United States Geological Survey(USGS) 21 class land use image of VHR remote sensing dataset which have been widely used in the world, and the SPOT 5 satellite remote sensing data of Shanghai city are used for supervised classification. We compare the proposed algorithm with other typical linear dimension reduction methods such as LDA and PCA, also with other canonical manifold learning dimension reduction methods such as ISOMap, LLE, Supervised LLE and multiple manifolds learning. The experimental results demonstrate the feasibility and effectiveness of our proposed algorithm. In addition, the proposed HML is applied to supervise classificaiton of the other data sets, i.e., the UCI data sets, and the experimental result also demonstrates the similar effectiveness.