An Experiment Study On Stabilization For A New Boosting Algorithm

Bagging and Boosting are the most known ensemble algorithm, and Both algorithm have been wildly used as their excellent performance. Boosting algorithm has a significant drawback due to its concentrating on hard instances. Thus ensemble developed by Boosting will be easy to get paralyzed after several iterations which made the ensemble result unstable. However, Bagging algorithm generated individuals networks independently, so the "difficult" samples could not be accumulated, but Bagging algorithm is not targeted to the individual training network, this makes the accuracy of the method could not be controlled.This paper proposed a new Boosting algorithm called BSLB (Boosting Seeded Local Bagging) which based on Boosting and combined with Bagging algorithm, in order to improve the stability of neural network ensemble algorithm, avoid those difficult samples accumulating while training the individual networks, and make the accuracy can be controlled. New algorithm calculated the error with local method to distinguish the "difficult" and "easy" samples and selected those "difficult" samples as "Seeds" which are used to generate new training sample set with Lazy method. This method provide a training sample set with good quantities to ensure the stability of the ensemble algorithm and focused on the individual networks training which make sure the result has a high accuracy.Experiment results show that BSLB has less dependence on individual network than Boosting and Local Boosting method which make the result more stable and robust. Especially, BSLB has a lower correlation than Boosting in noise case. The generalization result is more accurate than AdaBoosting and Bagging method, and the network structure is more stable than Local Boosting algorithms. The relevance of BSLB algorithm is much lower than Local Boosting which makes the impact of unstable fact less than others. Although much more time will be needed to train each base classifier, total time expense will be lower than Boosting because BSLB can be parallel computed as Bagging.