A Multi-label Classification Algorithm Based On Subspace Decomposition With Discriminant Analysis

Multi-label classification algorithm deals with the classification problem, in which each instance belongs to multi-classes at the same time. As one of the very forefront research topic in machine learning, multi-label classification algorithms have varieties of applications in our daily life, like text categorization, image annotation and medical inference etc. Multi-label classification is a more general classification problem compared with traditional classification algorithms. On the other hand, the complexity of the algorithm would grow exponentially with the growth of the label number. So there are still a lot of challenges and difficulties in multi-label classification problem.Up to now, multi-label classification algorithms which base on the correlations between the labels have some common disadvantages: Firstly, performance of the low-order multi-label classification algorithms would decline as the correlations between labels become strong; Secondly, the complexity of the high-order multi-label algorithms would grow rapidly(exponentially) with the growth of the label number.In order to overcome the weakness of present algorithms, subspace decomposition method and linear discriminant analysis algorithm are taken into account in developing the suggested multi-label classification model. The contributions of this paper could be concluded as:First, multi-label data are represented as the sum of components from each label by subspace decomposition method, which decouples the correlations between labels and reduces the complexity of the multi-label classification problem from exponential complexity to linear one.Second, discriminant information is calculated by conducting linear discriminant analysis algorithm on each subspace, which helps to construct the suggested multi-label classification model satisfying the discriminant criterion and consistency constrain. At the same time, a gradient descent cycling iteration method is adopted to solve the variables in the model.Third, a more general version of the proposed multi-label classification model is deduced with mathematic description from the presented one based on the intrinsic demands of multi-label classification algorithms, while experiments on several datasets compared with different algorithms have demonstrated the efficiency of our multi-label classification algorithm.