Label Structure Based Deep Learning For Long-tail Distributed Classification

The long-tail distribution is an important topic in the area of big data,where the most classes occupy only a small number of samples.These minority classes cannot be ignored and sometimes are even more valuable,because samples from these categories make up an important part of the whole dataset.The key issue of long-tail distribution is how to manage the insufficiency of instances and the failure to describe the intraclass diversity for the minority classes.Some existing approaches have been proposed to deal with the imbalance problem.However,most of them concentrate on enhancing the statistical ability and fail to recognize samples beyond the description of the training data.In this thesis,the main idea is how to extend the intra-class boundaries of the minority classes with the knowledge implied in the relationships among different categories.Based on the one order linear label relationship and the high order label hierarchical correlation,the effectiveness of long-tail classification when introducing the label structure is discussed.The one order linear label relationship offers an additional description about the label.Specifically,there are three research topics as follows:(1)One Order Linear Label Relationship Based Deep modelBased the one order linear relationship,the probability relationship among different categories can be introduced as another knowledge to enhance the description of the label.Thus,the deep neural network and the conditional random field are combined to extract more discriminative features and model the linear relationship between different classes.(2)High Order Label Hierarchical Correlation Based Deep Collaborative Multi-Task LearningTo make use of the hierarchical label structure,a top-down strategy is usually adopted.However,errors in the parent nodes will be propagated to the children nodes in this strategy.Therefore,a deep collaborative multi-task learning model is designed to combat these problems,where each classification problem along the hierarchy is treated as a classification task.In this model,any two tasks can affect each other with the relationship extracted from the tree-shape label structure in both training and test procedures.(3)Label Structure Construction with Deep LearningTo make these label structure based methods applicable to any tasks without inherent label relationship and obtain a label structure that benefits the classification procedure,a deep super-class learning model is designed.Motivated by the observation that classes belonging to the same super-class usually have more similar evaluations on the features than those belonging to different super-classes,a deep super-class learning(DSCL)model is proposed,where a block-structured sparse constraint is designed and attached on the top of a convolutional neural network.Thus,the proposed DSCL model can accomplish representation learning,classifier training,and super-class construction in a unified end-to-end learning procedure.Finally,this thesis proposes a set of solutions for the long-tail distribution problem based on the label structures.It can be concluded that it is effective to mine and exploit the relationships among different categories to solve the long-tail classification problem.