| Deep learning has powerful feature learning capability and nonlinear approximation characteristic,which can learn multi-level abstract features from data.However,most deep models use gradient-based optimization algorithms to update model parameters iteratively,and these models have more complicated network structure,slower training speed and larger number of network parameters.Stochastic Configuration Network（SCN）assigns node weights and biases in the hidden layer using inequality constraints,and constructs the network structure incrementally,which guarantees the universal approximation of the model.Since it was proposed in 2017,SCN has been rapidly promoted and developed in the fields of hardware implementation,computer vision,medical data analysis,fault detection and diagnosis,system modeling and prediction.Although SCN can be extended to a deep version（Deep SCN）with a multi-layer structure,its generalization and robustness still need to be improved.In particular,the feature representation,robust modeling,hyper-parameter setting and parameter learning of the model need to be further researched and improved.To solve the above problems,a systematic and in-depth study on feature optimization and ensemble,hyper-parameter/parameter optimization and ensemble are carried out in this thesis,the contents are as follows:1.Research on stacked deep stochastic configuration network based on feature optimization and ensemble.To enhance the feature learning capability of the SCN,this thesis proposes a stacked deep stochastic configuration network based on sparse feature ensemble by stacking random sparse autoencoder and SCN.Specifically,effective sparse feature representation can be extracted through the feature optimization and ensemble strategy based on random sparse autoencoder.Then the ensemble optimized features are used for SCN training to execute the supervised learning task,thus improving the model learning accuracy.The random sparse autoencoder-based feature optimization and ensemble can guarantee the learning efficiency and provide an option for the feature learning of Deep SCN and its variants.2.Research on weighted deep stochastic configuration network based on sparse feature ensemble.In practical application scenarios,noise and outliers in the data tend to reduce the robustness of the model.To improve the uncertain data regression performance of Deep SCN,this thesis proposes weighted deep stochastic configuration network based on M-estimator function.A stochastic configuration sparse autoencoder based on the supervision mechanism is designed for sparse feature learning and ensemble to improve the feature learning capability of Deep SCN.Based on the traditional Deep SCN,two commonly M-estimator functions（i.e.,Huber and Bisquare）and L2 regularization strategy are introduced into objective function to update the model parameters by weighting the training sample errors,which reduces the negative impact of outliers and improves the robustness.3.Research on hyper-parameter optimization of deep stochastic configuration network ensemble.To select scale of node parameters adaptively and improve the generalization of multi-model ensemble based on Deep SCN,this thesis proposes a deep stochastic configuration network ensemble model based on chaotic sparrow search algorithm.In the first place,chaotic sparrow search algorithm is designed which mainly utilizes logistic mapping,self-adaptive hyper-parameters,mutation operator to enhance the global optimization capability.In the second place,the chaotic sparrow search algorithm is used to optimize the hyper-parameter of Deep SCN,then we use optimal hyper-parameter to training the base models and construct the adaptive boosting model,which reduces the model generalization error and improves the ensemble model regression accuracy.4.Research on decorrelated deep stochastic configuration network ensemble based on greedy strategy.As a random neural network model,the parameter learning of Deep SCN is relatively inadequate.To optimize the parameter learning process of Deep SCN based on the supervision mechanism,guarantee the individual performance and diversity of the base models,this thesis proposes a deep greedy stochastic configuration network ensemble model based on negative correlation learning.Firstly,a greedy optimization strategy based on artificial rabbits optimization is designed under the supervision mechanism of Deep SCN to configure node parameters by minimizing model residuals.Secondly,the greedy strategy is used to assign parameters of base models,we employ the negative correlation learning strategy to trade-off the bias,variance and covariance of ensemble model,which reduces the correlation among base models,thus improving the generalization performance of the ensemble model.This thesis has 30 figures,53 tables,and 215 references. |
