Exploiting Label Relationships In Multi-label Classification

Multi-label learning is critical in many real world application domains including text classification,image annotation,video semantic annotation,gene function analysis and etc.Recently,multi-label learning has attracted intensive attention and generated a hot research topic in machine learning community.Meanwhile,multi-label learning has remarkable advances achieved in both academia and industry.However,multi-label learning remains a challenging task due to the large number of variations caused by the number of labels,label relationships,missing label and so on.Label relationships is a complex and important factor,which plays an critical role in improving the perfonnance of multi-label classification.Meanwhile,label relationships's effective learning will enrich the meaning of data representation.Therefore,multi-label classification is mainly faced with the following challenges:there are correlations among different labels,and the correlations are quite different;with the number of labels increases,the dependencies between labels become more complex and face the challenge of time and space complexity;in addition,the application of label relationships in the problem of missing label.In this paper,two kinds of multi-label classification models are proposed to solve above problems,the main contributions are summarized as follow:Firstly,we proposed a NN_AD_Omega model via neural network for exploring global labels dependencies.Considering the differences of label dependencies among different labels,this paper constructs the global label relationships matrix to characterize the labels dependencies.The matrix is symmetric matrix whose diagonal elements indicate that each label has the strongest dependency on itself.NN_AD_Omega model introduces the global label relationships matrix in the top layer of the neural network,which enhances knowledge sharing between labels at the output layer.As a good by-product,the learnt label correlations has ability to improve prediction performance when the instances,partial labels are missing because they can capture the intrinsic structure among data and compensate for errors caused by missing labels.Experiments on four real multi-label datasets demonstrate that the proposed method can exploit the label correlations and handle missing label data,and obtain promising and better label prediction results.Secondly,we proposed a BooMF_LLDA model based on supervised topic model to explore local label dependencies.With the number of labels increases,the size of the global label relationships matrix is becoming larger and larger,and the time and space complexity of updating the matrix is getting higher and higher.In order to reduce the time and space complexity of label relationships matrix,this paper constructs the local label relationships matrix to characterize the labels dependencies.The matrix is obtained by Boolean matrix factorisation of the label matrix,and the implicit label representation of the label matrix is also obtained.BooMF_LLDA model applies the implicit label representation to supervised topic model as supervisory information of latent topic assignment during training phase.Supervised topic model constrains Latent Dirichlet Allocation by defining a one-to-one correspondence between topic model's latent topics and instance implicit labels.Experiments on two real multi-label datasets demonstrate that the proposed method can exploit the label correlations,and obtain promising and better label prediction results.