An Ensemble Learning Algorithm Based On The Feature Extraction By Wavelet Transform

Ensemble learning is the first of the four major researches in machine learning by T. G. Dietterich. Generalization ability is the principle issue in the field of machine learning. Ensemble learning is a learning paradigm, which can improve the generalization ability of the algorithm by combining classification algorithm through a strategy. Good generalization ability of base classifier and diversity between the base classifiers is the key to successful ensemble learning. Wavelet transform as an effective feature extraction method get more and more attention by researchers in recent years, because of its good local properties in the time domain and frequency domain.How to design an ensemble learning algorithm with good generalization ability has been a hot topic of ensemble learning research. Based on depth analysis and study classical ensemble learning algorithm, this paper presents an ensemble learning algorithm based on the feature extraction by wavelet transform,which is called Wavelet-Forests. The main idea of this algorithm is to improve the generalization ability of the ensemble learning algorithm by the diversity between the base classifiers. In the process of base classifier construction,Wavelet-Forests adapts combining different subsets of training data and different feature subset. To create the training data for a base classifier, the feature set is randomly split into n subsets. According to a breakdown of each feature subset selected a data subset by a non-empty class subset, then a subset of these data using bootstrap to generate a new subset of data. Wavelet is applied to each subset, get a wavelet coefficient matrix of data subset corresponding to. Integrated wavelet coefficients matrix, get the base classifier training data sets. The class can be obtained by calculating the average weight of each class of the base classifier,and then back to the class with maximum weight.The common platform WEKA has been used to validate the performance of Wavelet-Forests algorithm. We examined the algorithm on the UCI repository and compared it with J48, the classical ensemble learning algorithm Bagging, AdaBoost and Random Forest. The experimental results were analyzed by accuracy and ROC curve, which were favorable to Wavelet-Forests. The results show that, Wavelet-Forests has high accuracy, and can be used to deal with class imbalance problem.