Research On Multi-label Classification Algorithm And Its Application In Compound Facial Expression Recognition

Multi-label classification aims to accurately classify different objects in the same image,which is widely used in text classification,image recognition and other practical scenes.There are two core problems in the study of multi-label classification.First,how to use the correlation between multi-label categories to infer other possible categories through the classified sample labels;second,the number of samples in different categories in the classification data set usually varies greatly,which causes the problem of category imbalance.To solve these two problems,we propose a new multi-label classification model named GMLC(Graph Net for Multi-label Classification).In this model,the graph convolution network is introduced to learn the dependence between different labels,and the problem of class imbalance is alleviated by the weight loss function under the number of effective samples.In addition,we find another very challenging task,compound expression recognition,which is also a multi-label classification problem in essence.Therefore,we improved the GMLC model and applied it to compound expression recognition,and achieved excellent recognition results.The main work and innovation of this paper are summarized as follows:(1)In order to more accurately mine the correlation between tags,we use graph structure to build the dependency between multiple tags.Specifically,we regard the nodes of the graph as category labels and the edges as dependencies between labels.On this basis,the semantic information of the tag is spread among different nodes through the graph convolution network,and the output of the last layer of graph convolution is regarded as the classifier of tag correlation.These classifiers are applied to the global image features extracted by the feature learning module to help the model better complete the classification task,thus forming an end-to-end network system named GMLC.(2)In order to solve the problem of class imbalance in the task of multi-label classification.We start with the loss function of the model,and uses the idea of effective sample number to remove the approximate samples in the original data set that have no practical effect on model optimization.On this basis,the common cross entropy function is improved,and each category is reweighted according to the number of effective samples,so as to improve the contribution of a few categories to the model.(3)In compound expression recognition,a sample image corresponds to multiple category labels at the same time,such as surprise(happy + surprised),which is the problem to be solved by multi-label classification.Based on the above findings,we believe that compound expression recognition is essentially a multi-label classification problem,and innovatively improve the GMLC model proposed in this paper and introduce it into compound expression recognition.The graph convolution network is used to mine the dependence of different basic expressions in the semantic space,and the second-order covariance pooling is used to capture the distortion of facial muscles,so as to further improve the recognition accuracy.Experimental results show that our model has achieved good results and is superior to the existing model in most evaluation metrics,no matter in the traditional multi-label classification scene or in the task of compound expression recognition.