The Research Of Ensemble Learning And Its Application Based On Feature Selection

Ensemble learning is a learning paradigm, in which a collection of a finite number of individual is trained for the same task. Recently, it has become a hot research topic in the machine learning community because of its high generalization ability. Therefore, generalization ability is the principle issue in the field of machine learning. Feature selection for ensembles has shown to be an effective strategy for improving the generalization ability of the ensemble.Nowadays, accuracy, speed and understanding are the three main topics in current machine learning community. As a pre-processing step, feature selection is one of the techniques which can improve the accuracy, speed and understanding at the same time. In data mining field, feature selection can remove many irrelevant features and improve the speed and accuracy of data mining. Feature selection chan select the most relevant features to help understand the problems from chemistry, medicine and biology fields. In general, feature selection is a key technique to improve the accuracy, speed and understanding of intelligent systems.A typical feature selection process can be viewed as a composition optimization process in principle, which selects a part of the original features to make some specific evaluation function optimal. Search is the main techniques to solve the optimization problems, in which the feature subset search method decides the search procedure and the evaluation criteria is another key factor.In this paper, we focused on how to improve the generalization ability of the ensemble learning using proper feature selection algorithms. The main work and creative points in this paper are listed as follows.(1) The research on the feature selection of individual in the ensemble learning has been done. Two typical feature selection approaches namely the PRIFEB (Prediction RIsk based Feature sElection for Bagging) and MIFEB (Mutual Information based Feature sElection for Bagging) are proposed to improve the generalization ability of the ensemble learning.(2) Multi-task learning techniques can employ the removed redundant information to improve prediction accuracy. Three new algorithms named H-MTL(Heuristic Multi-Task Learning)and GA-MTL (Genetic Algorithm based Multi-Task Learning) as well as GA-ENMTL (Genetic Algorithm based Ensemble Multi-Task Learning) are proposed to improve the generalization ability. (3) Combining the mutual information criteria with ensemble techniques, MISEN algorithm (Mutual Information based Selective ENsemble) is proposed to apply mutual information criteria in the selective ensemble techniques instead of genetic algorithm.(4) Feature selection based semi-supervised learning method has been investigated. A Feature Selection Co-Training (FESCOT) algorithm is proposed to improve the generalization ability of semi-supervised learning. Experimental results show that FESCOT can help to remove the irrelevant and redundant features which will damage the performance of the co-training method.