A multi-class classification using ensembles of multinomial logistic regression models

This research proposes a method for multi-way classification problems using ensembles of multinomial logistic regression models. A multinomial logit model is used as a base classifier in ensembles from random partitions of predictors. The multinomial logit model can be applied to each mutually exclusive subset of the feature space without variable selection. By combining multiple models the proposed method can handle a huge database without a parametric constraint needed for analyzing high-dimensional data, and the random partition can improve the prediction accuracy by reducing the correlation among base classifiers. The proposed method is implemented using R and the performance including overall prediction accuracy, sensitivity, and specificity for each category is evaluated on real data sets and simulation data sets. To investigate the quality of prediction in terms of sensitivity and specificity, area under the ROC curve (AUC) is also examined. Performance of the proposed model is compared to a single multinomial logit model and another ensemble method combining multinomial logit models using the algorithm of Random Forest. The proposed model shows a substantial improvement in overall prediction accuracy over a multinomial logit model.