Improving generalization capability of neural networks under conditions of sparse data: A new committee formation approach

This dissertation attempts to improve the generalization capability of committee networks under sparse data conditions. The committees are formed based on a linear combination of neural networks using the concepts of ridge regression, principal component regression, and the r-k class estimator. Here a set of trained bootstrap networks serve as an input variable and the target response is the dependent variable in the regression models. The experimental results suggest improvement in models' generalization capability of the proposed algorithms when compared to that of a single network and the bootstrap committee fused using the simple average. By automatically and properly selecting the tuning parameters, the proposed algorithms can integrate unique and useful knowledge from the committee members as well as effectively reduce multicollinearity effects, and thus perform robustly in all kinds of sparse data conditions. The recommend method can also be applied to non-sparse data cases.