Feature Selection Algorithms Based On Multi-objective Optimization For Multi-label Classification

Classification denotes that labels of the unknown samples are predicted by a model which is trained on some known samples. It’s divided into single-label and multi-label classification according to the number of labels on samples. Classification tasks that naturally emerge in multi-label domains, such as text categorization, automatic Scene annotation, and gene function prediction, have been widely applied. The performance of multi-label classification is strongly influenced by the quality of features. Theoretically, the irrelevant or redundant features decrease the similarity of each pair of patterns in same class and lead to poor performance of classification. Therefore, feature selection plays an important role in multi-label classification. Currently, there are two groups of methods dealing with multi-label feature selection problem:the filter methods and the wrapper methods.There are three categories of evaluation criteria respectively based on samples、labels and ranking for multi-label classification, and it exists contradictions among those criteria. The goal of feature selection is to retain or even improve the performance of classifier. Therefore, multi-label feature selection is essentially a problem of multi-objective optimization. In this paper, we propose a wrapper feature selection method based on evolutionary multi-objective optimization algorithm (NSGA-Ⅱ) for multi-label classification. The main idea of the proposed method is:we build two objective functions using the evaluation criteria of multi-label classification, i.e., maximizing average precision function and minimizing hamming-losing function. To achieve optimal feature subset, we simultaneously optimize those two functions using NSGA-Ⅱ.In experiment of algorithm convergence, we tested the proposed method with Yeast and Emotions data sets. The results demonstrate that the proposed method has good convergence. In comparison of algorithms, we compared performance of our proposed method against other three conventional multi-label feature selection methods based on nine evaluation criteria of multi-label classification and eight data sets. The experimental results show that our proposed method is top on six evaluation criteria and is second on the other three criteria. At the same time, we compared dimension of optimal feature subsets yield by those feature selection methods. And results show that our proposed method can obtain smaller feature subsets when it effectively improves performance of ML-kNN.