| With the rapid development of the data collection and storage, high-dimensional data such as spaceflight remote sensing, biology data, network data and money market dataemerged recently. How to express these high-dimensional data in the low-dimensional space and discover the intrinsic structure is an important topic of machine learning and pattern recognition. Over the past decades, a large family of methods has been designed to provide different solutions to the problem of dimensionality reduction. They are widely used in image analysis and processing, multimedia processing, medical data analysis, climate forecasting, computer vision and cross lingual text classification, etc..Pattern recognition mainly include the research of classification and clustering, and classification can be divide into supervised classification and unsupervised classification, this paper mainly research the method of supervised classification. All data sets used in this paper have a dimensional higher than 50. We researched the classification of high-dimensional data based on the theory of partial least squares method, and this paper mainly includes four parts:(1) Introduce the theory of PC A and PLS from the perspective of feature extraction, by introduce the theory of PCA to help get a deep understanding of PLS,then can know how PLS achieves data dimensionality reduce in the process of high-dimensional data classification.(2) One disadvantage of PLS is in directions in X space having large variance unrelated to Y. the PLS model may not work well. The second part of this paper added the algorithm of SLT (Slice transform) into the process of PLS, and combined with the method of discriminant analysis to achieve the design of PLSDA and SLT-PLSDA to get classification models. In order to compare the process of PLSDA and SLT-PLSDA and the capacity of classification, we used one data set do experiments.(3) We proposed using cross-validation method to evaluate the performance of the classification model, and then worked out the model of cross-validation. Five data sets was tested by the model and then the error rates of PLSDA and SLT-PLSDA models were acquired, then the evaluate result came out. These values not only stand for the ability of PLSDA and SLT-PLSDA models, but can reveal which model is better.(4) Because the exists of the process to select the number of latent variables, so the training of classification faces with the risk of falling into local minimum, and the risk is bring by the random selection of training data set. So combining the cross-validation method, we proposed two methods to get the classification model combining with integrate learning. One is get different results by different classification models and use the method of voting to classify unknown samples. The other is to get different classification models by cross-validation and achieve the accuracy of each model by cross-validation, and then combine all the models into one base on these accuracies. By these two methods, not only the risk of local minimum can be avoided, but also the classification accuracy improved and the stability of the model improved. |
