A comparison of methods for learning cost-sensitive classifiers

There is a significant body of research in machine learning addressing techniques for performing classification problems where the sole objective is to minimize the error rate (i.e., the costs of misclassification are assumed to be symmetric). More recent research has proposed a variety of approaches to attacking classification problem domains where the costs of misclassification are not uniform. Many of these approaches make algorithm-specific modifications to algorithms that previously focused only on minimizing the error rate. Other approaches have resulted in general methods that transform an arbitrary error-rate focused classifier into a cost-sensitive classifier. While the research has demonstrated the success of many of these general approaches in improving the performance of arbitrary algorithms compared to their cost-insensitive contemporaries, there has been relatively little examination of how well they perform relative to one another. We describe and categorize three general methods of converting a cost-sensitive method into the cost-insensitive problem domain. Each method is capable of example-based cost-sensitive classification. We then present an empirical comparison of their performance when applied to the KDD98 and DMEF2 data sets. We present results showing that costing, a technique that uses the misclassification cost of individual examples to create re-weighted training data subsets, appears to outperform alternative methods when applied to DMEF2 data using increased number of re-sampled subsets. However, the performance of all methods is not statistically differentiable across either data set.