Research On Classifier Ensemble

Ensemble learning constitutes one of the main current directions in machine learning research. Ensemble methods are learning algorithms that construct a set of classifiers and then classify new data points by taking a weighted vote of their predictions. Ensemble learning has been applied to a wide range of real problems. Ensemble learning is primarily used to improve the performance of a model, or reduce the likelihood of an unfortunate selection of a poor one. Other applications of ensemble learning include assigning a confidence to the decision made by the model, selecting optimal (or near optimal) features, data fusion, incremental learning, nonstationary learning and error-correcting. Despite of the absence of an unified theory on ensembles, there are many theoretical reasons for combining multiple classifiers, and an empirical evidence of the effectiveness of this approach. This paper focus on:Design of learning label in Ensemble learning, analysis of ensemble error, fast selective ensemble, the detection of outlier based on revised Boosting algorithm, segment of model space based on clustering, ensemble learning based on ACS (Adaptive Clustering Sampling), and incremental tracking based on selective ensemble.â… .The theorem that n+1 symmetry vector can be constructed in n-D space was proved, and the vector label was design based on it. In this way, the unity of the majority voting and average method is achieved. Using vector label, the ensemble algorithm, which can solve two-class problem, can solve multiple-class problem without any change. The development of Korgh's theory about ensemble error in 1995 was done, and we proved that the diversity and the performance of base classifiers must be concerned together by experiment and the theory mentioned before. Two algorithms, Sort-Bagging and Random-Bagging, were designed to enhance the performance of Bagging (Bootstrap aggregating).â…¡.The agreement of base-classifiers was researched to design new algorithm on selective ensemble. Based on agreement and diversity of base-classifiers, a new hierachical Bagging pruning techniques will be mentioned in this paper. The new algorithm's runtime of train is much faster than GASEN (Genetic Algorithm Selective ENsemble) and CLU_ENN (CLUstering Ensembles of Neural Network), and the new algorithm could be supported by parallel computing.III.The technique of using Boosting to detect outliers was reformed in this paper. Based on clustering technique, two fast Boosting algorithms were designed out. One is using Boosting and clustering to pruning Bagging, the other is using clustering to speed AdaBoost(Adaptive Boost). Based on the analysis of detection of outliers by Boosting, we put forward the concepts of pseudo-outlier and weak-boosting. Finally, the new algorithm, which combines Boosting, Cascade technique, and K-NN, was proved to be excellent in detecting outliers in artificial data sets.â…£.The ideas of "Divide and conquer" and "Collage" were used to explain why ensembles can often perform better than any single classifier. And the technique about using clustering to segment model space was designed to solve the "divide" problem. Compared with the traditional clustering, the clustering mentioned in this paper is on the opposite approach. The ACS was revised to adapt machine learning, and this can make the base classifiers take active learning on samples. We combine the ACS and vector label to solve "conquer" problem and can control the learning process of weak classifiers. Finally, the mean algorithm was used to solve the "collage" problem. By adjusting the parameters, we could attain different intensity of learning and different noise suppression. The experiment also described the relationship between the noise suppression and learning ability is irreconcilable.â…¤. From the perspective of machine learning, tracking task is essentially incremental learning. The use of selective ensemble achieves incremental learning in tracking task. The detection and remove of noise pixels could be completed with tracking at the same time. According to the characteristics of the tracking task, the technique of background prediction was designed and applied in the experiment of tracking. The desired results were achieved by applying local Walsh HSV texture features.