Research On Multi-dimensional Classification Approaches Via Class Dependencies Modeling

In traditional supervised learning,one popular learning task is to train classification models supervised by one class variable,e.g.,multi-class classification.However,in many real-world applications,the simplifying assumption that each example is associated with only one class variable does not fit well,because it is hard to use single class variable to characterize object’s rich semantic information that is usually multi-dimensional.To deal with this kind of problems,one natural solution is to associate multiple class variables with the object,which yields the multi-dimensional classification(MDC)framework.Compared to multi-class classification,each MDC example is also represented by a single instance while associated with multiple class variables.Here,each class variable corresponds to one heterogeneous class space characterizing the object’s semantics from one dimension.It has been shown that how to effectively model the dependencies among class variables(i.e.,heterogeneous class spaces)is the key to solve the MDC problem.This dissertation focuses on designing novel MDC approaches by modeling the dependencies among class variables with either explicit or implicit strategies.The main contributions of this dissertation are summarized as follows:On one hand,for traditional explicit strategy,we propose the MDC approach named M3 MDC based on statistical machine learning techniques which explicitly models the dependencies among class variables via covariance regularization.As it is very difficult to model the dependencies among class variables with one single structure due to the huge class combinations in output space and limited training examples,we further propose the MDC approach named SEEM which models the dependencies among class variables via stacked dependency exploitation.(1)The M3 MDC approach learns classification model for each pair of class labels in individual class space with maximum margin-based techniques and considers the dependencies among class variables with covariance regularization.The derived optimization objective function is convex to each set of parameters and can be solved via alternating optimization admitting quadratic programming or closed-form solution in either alternating step.Experimental results show that M3 MDC achieves superior performance against compared approaches and also recovers the dependencies among class variables.(2)The SEEM approach solves the MDC problem with a two-level deterministic problem transformation strategy and gradually considers low and high order dependencies among class variables via stacked dependency exploitation.Specifically,SEEM considers the second order dependencies which can be modeled more reliably with limited training examples in the first level,and then stacks related predictions from the first level for each dimension to further consider the high order dependencies among class variables in the second level.Experimental results show the superiority of SEEM against compared approaches and validate the effectiveness of stacked dependency exploitation strategy in SEEM.On the other hand,as it is very difficult to explicitly model the dependencies among class variables in the original output space due to the heterogeneity of class spaces,we further attempt to implicitly model the dependencies among class variables via manipulating output space and input space,respectively.We propose MDC approaches named SLEM and DLEM by manipulating output space via label encoding techniques and propose the MDC approach named KRAM by manipulating input space via feature augmentation mechanism.(3)The SLEM approach encodes the output space into another one to alleviate the heterogeneity of class spaces via sparse label encoding and learns predictive model in encoded space to consider the dependencies among class variables.SLEM works in an encoding-trainingdecoding framework.The encoding phase transforms label space into another one via pairwise grouping,one-hot conversion and sparse linear encoding while the decoding phase can be done via adapting the orthogonal matching pursuit algorithm.Experimental results show the superiority of SLEM against compared approaches and validate the effectiveness of its encoding strategy.(4)For the two issues of SLEM that feature information is ignored in encoding phase and sparse reconstruction procedure exists in decoding phase,the DLEM approach transforms the output space into another one via decomposed label encoding and labeling information enrichment to utilize the helpful information from feature space in encoding phase.After the predictive model has been trained in encoded label space,the decoding phase of DLEM can be directly done according to the decomposition rule.Experimental results show the superiority of DLEM against compared approaches and validate the effectiveness of its encoding strategy.(5)The KRAM approach solves the MDC problem with feature augmentation strategy and investigates the feasibility of modeling the dependencies among class variables via feature space manipulation.Specifically,KRAM generates an augmented feature vector for each example by making use of k NN techniques,and then brings the discriminative information from class spaces which might be helpful for MDC model induction into feature space via feature augmentation.Finally,the MDC model is trained over the augmented feature space.Experimental results show that the generated augmented features by KRAM significantly improve the generalization performance of existing MDC approaches.