Boosting Algorithms Via Margin Theory

Margin theory is crucial to the generalization analysis of Boosting algorithms,as a result,Boosting algorithms via margin theory becomes an important research issue.This thesis first define the k*-optimization margin distribution that is approximately optimal.Compared with the margin distribution generated by AdaBoost,it has sharper generalization error bound,and thus lead to better generalization ability.On this basis,we propose two strategies,KM-Boosting and MD-Boosting,to approach the k*-optimization margin distribution.Next,aiming at the drawback that the existing margin-based generalization bounds are difficult to compute,we derive a moment generalization bound,which is a generalization error bound for Boosting in terms of the first and secondary moment of the margin distribution,and reveals the closer relationship between margin distribution and generalization error.Then we present a moment criterion for Boosting model selection,which is based on the moment generalization bound and obtained by maximizing the first moment and minimizing the second moment of the margin distribution.We further develop a moment-based Boosting algorithm by adopting column generation to implement the criteria.Finally,Experimental results show our methods are sound and efficient,which consistent with the theoretical analysis.