Category space dimensionality reduction for supervised learning

In this research, we investigate the reasons that make the multiclass classification problem difficult and suggest that category ambiguity is at the heart of the problem. We analyze previous efforts at multiclass classification and explain how they do not account for this ambiguity when they suggest simple voting schemes offer the combination of pairwise binary classifiers as solutions. Then we demonstrate that most other methods lack the notion of a geometric space to represent the classes. These essential concepts have been neglected because in the past, classes were perceived to be independent. This leads to limited approaches in which techniques assume distinct classes and assign nominal labels to them. We argue that class relationships exist that must be exploited. The approach we propose gives an alternate method for dimensionality reduction so that multiclass classification techniques can overcome several of the problems that exist with pairwise classification schemes and exhibits better performance on many problems. We look to separate objects according to similar qualities and characteristics by projecting to a space---a Category Space---defined by the number of classes and the properties of the classes. After dimensionality reduction to the category space, we use a classification technique to evaluate performance on a large collection of benchmark data sets. Finally, we detail the strengths of our approach, and provide a framework for alternative objective functions for linear and kernel-based projections. Our contributions span the range of dimensionality reduction, classification, and optimization for multiclass problems.