Optimal Margin Distribution Machine

The classical large margin based methods in statistical learning try to maximize the minimum margin,however,recent theoretical studies on Boosting disclosed that maximizing the minimum margin does not necessarily lead to better generalization per-formances,and instead,the margin distribution is more crucial,so how to efficiently optimize the margin distribution turns to be a new challenge in machine learning.For four commonly encountered learning tasks,this paper proposes four methods respec-tively which can explicitly optimize the margin distribution,so as to completely set up this new statistical learning paradigm,including the following aspects,1.Binary optimal margin distribution machine,ODM.For the binary classification problem,this paper tries to optimize the margin distribution by maximizing the margin mean and minimizing the margin variance simultaneously,and proposes a dual coordinate descent method to solve the final optimization problem.The empirical studies show that ODM is better and comparable at worst case than the minimum margin based binary classification methods.2.Multi-classification optimal margin distribution machine,mcODM.For the multi-classification problem,this paper optimizes the margin distribution by defin-ing the multi-class margin as the minimum of the binary margin between each class and all the other classes,and proposes a dual block coordinate descent method to solve the final optimization problem.The empirical studies show that mcODM is better and comparable at worst case than the minimum margin based multi-classification methods.3.Optimal margin distribution clustering,ODMC.For the clustering problem,this paper tries to find a hyperplane,such that when labels are assigned to different clusters according to this hyperplane it can also achieve optimal margin distribution,and proposes a stochastic mirror descent method to solve the final optimization problem.The empirical studies show that ODMC is better and comparable at worst case than the minimum margin based clustering methods.4.Semi-supervised optimal margin distribution machine,ssODM.For the semi-supervised learning problem,this paper tries to find a hyperplane,such that when labels are assigned to each unlabeled instance according to this hyperplane it can also achieve optimal margin distribution,and proposes a stochastic mirror prox method to solve the final optimization problem.The empirical studies show that ssODM is better and comparable at worst case than the minimum margin based semi-supervised learning methods.