Neural-Network Construction And Selection In Nonlinear Modeling

For any modeling problem, it is very important to be able to estimate the reliability of a given model. This paper presents an economic alternative to the construction of CIs using neural networks. This approach being built on the linear least squares (LS) theory applied to the linear Taylor expansion (LTE) of the output of nonlinear models, we first recall how to establish CIs for linear models, and then approximate CIs for nonlinear models. We exploit these known theoretical results for practical modeling problems involving neural models. We show that the LTE of a nonlinear model output not only provides a CI at any input value of interest, but also gives a tool to detect the possible ill-conditioning of the model, and estimates its performance through an approximate leave-one-out (LOO) score. Base on CI of neural model, we study how statistical tools which are commonly used independently can advantageously be exploited together in order to improve neural network estimation and selection in nonlinear static modeling. The tools we consider are the analysis of the numerical conditioning of the neural network candidates, statistical hypothesis tests, cross validation, and so on. We present and analyze each of these tools in order to justify at what stage of a construction and selection procedure they can be most useful. On the basis of this analysis, we then propose a novel and systematic construction and selection procedure for neural modeling. We finally illustrate its efficiency through simulations. As the results of the study are promising, it is suggested that a statistical analysis should become an integral part of neural network modeling.