Characteristic Transformation In Classifier Combination

As an effective method to enhance the performance of weak learner, classifier combination method was developed in machine learning first. It is considered as one of the best learning methods by the last ten years. Plenty of principles and experiments have proved that combination methods model has notable priority to simple methods model.To classifier combination methods, one of the important factors which influence the performance of classifier combination is the difference between these basic classifiers. Different methods are used in different combination algorithms to insure the difference. As the classic classifier combination algorithms Bagging and Boosting both are use the way of Random sampling to produce different base classifier, from the perspective of training samples. Random sampling can ensure different classifiers, but it can bring problems too. It exists the risk of loss of information, especially when the training sample set is small, the problem will be even more highlights.Inspired from the algorithm RotationForest, ensuring the differences between the basic classifications by the way of do some feature transformation to the training data set, we propose a new combining algorithm——ICATrees. Combined with the traditional classification methods, ICATrees are different:it establishes different basic classifiers from the perspective of attributes of data sets. By random division and feature transformation ICA, the training data set are mapped to a different feature spaces, then use the classification algorithm of decision tree J48 to learn the base classifier. The algorithm's learning is based on the complete set of training data, and thus can effectively prevent the risk of loss information caused by sampling. By random division and feature transformation ICA, the algorithm not only ensures that all the base classifiers are different., experiments on the 30 randomly selected UCI data sets also show that the algorithm can improve the classification accuracy to some degree, compared with Bagging and Boosting.