Progressive Ensemble Kernel-based Broad Learning System For Classification

Machine learning is widely used in various fields,and broad learning system(BLS)is a new kind of random feedforward neural network that helps feature representation and data classification in machine learning.In recent years,BLS has attracted the study and research of scholars because of the high efficiency and good generalization ability.The feature nodes of BLS are randomly mapped from the original features,and the enhancement nodes are mapped randomly again by feature nodes.The feature nodes and enhancement nodes are then concatenated together to form the hidden layer of BLS.After that,the weights of the output layer of BLS are determined by solving the pseudo-inverse.The calculation of the output weights of BLS does not depend on the back-propagation algorithm and the weights do not need to be updated iteratively,which brings better generalization and higher efficiency.However,BLS suffers from two drawbacks: 1)the classification performance depends heavily on the number of hidden nodes,which requires manually tuning;2)double random mappings bring about the uncertainty,which lead to increased sensitivity to noise data together with unpredictable impact on performance.In this paper,two improvement approaches are proposed to solve the problems of BLS.First,in order to improve the generation mechanism of enhancement nodes in BLS,a kernelbased broad learning system(KBLS)method is proposed by transforming the feature nodes obtained from the first random projection into a kernel space rather than a random feature space.This manipulation allows the reduction of uncertainty and contributes to performance improvements with the number of hidden nodes fixed,which indicates that manually tuning is no longer needed.The second approach is to further improve the stability and noise resistance of kernel-based BLS,therefore a progressive ensemble framework based on Boosting(Progressive ensemble kernel-based BLS,PEKB)is proposed.The residuals are calculated by gradient and sub-gradient of previous base classifiers and are used to train the following base classifiers.After the stop condition is reached,the results of all base classifiers are integrated to obtain the final result.We performed detailed complexity analysis and parameter analysis for the proposed algorithm and conduct comparative experiments against existing state-of-the-art random feedforward neural network methods on 20 standard data sets.Moreover,the application of proposed approaches over large scale data sets in the field of network intrusion detection is also carried out in the experiment part.Experimental results show that the method proposed in this paper has achieved good results in terms of accuracy and stability.