Nonlinear Time Series Modeling And Prediction Using Modified Neural Networks-Based Autoregressive Models

Time series are an integral part of our lives. They are found in the field of social and natural sciences, engineering and economy. The need for their analysis, modeling, prediction and control has become more urgent. Generally, the main objective of time series analysis is to develop models that can establish the relation between variables. To serve these purposes, various models ranging from linear to nonlinear have been developed throughout the time. Usually, when the relation among different time series variables is linear, linear models are more suitable. On the other hand, when this relation is nonlinear, nonlinear models are more suitable. Linear models are easier to identify and implement. However, most real world systems are governed by nonlinear rules, hence, the importance of nonlinear modeling technique in dealing with time series.In this thesis, the problematic of modeling and prediction of time series together with the control issues are considered. Four new models were developed including two fully nonlinear models that are enhanced type of neural networks and two more generalized models that are a combination of enhanced type of neural network with a linear model, thus having both a linear and nonlinear capability. Furthermore, a generalized nonlinear model is used as internal model to address the tracking issue of the real ship. The main work and innovations of this thesis are summarized as follows.Firstly, a special type of RBF namely the Local Variable-Weights RBF Network (LVW-RBFN) was developed to investigate the modeling and forecasting problem of nonlinear time series. The LVW-RBFN is a four layered RBFN comprising of input layer, hidden layer, weight layer and output layer. It is an enhanced type of RBF network, in which the constant weights that connect the hidden layer with the output layer in the standard RBFN are replaced by functions of the RBFN’s inputs computed via the weight layer. A structured nonlinear parameter estimation method (SNPOM) is applied to estimate the model. Case studies on several benchmark time series show that the prediction performance of the LVW-RBFN model is superior to other newly developed models including the RBF thus confirming that the proposed model is an enhanced type of RBF. Secondly, the modeling and prediction problems of nonlinear time series are addressed using radial basis function network with polynomial type weights (LPRBF) to approximate the functional coefficients of the general state-dependent autoregressive (SD-AR) model. The LPRBF differs from the RBF by its weight form; the LPRBF has polynomial type weights, which are dynamic and vary with the network input, while the weights of the RBF are constants they are not affected by the change in the input of the network. The LPRBF is a more generalized form of RBF. It may be considered as an improved version of the RBF. Thus the LPRBF.-AR can be regarded as an enhanced version of previous RBF-AR model and local linearly RBF based AR (LLRBF-AR) model. A fast-converging estimation method known as the structured nonlinear parameter optimization method (SNPOM) is applied to estimate the LPRBF-AR. Case studies on various time series and chaotic systems show that the LPRBF-AR modeling approach exhibits much better prediction accuracy compared with some other existing methods.Thirdly, an enhanced type of wavelet neural network, known as Local Polynomial Wavelet Neural Network with a Structured Nonlinear Parameter Optimization Method (LPWNN-SNPOM) is introduced. This is a five layered wavelet neural network comprising of input, output, wavelet, product and polynomial layers, which computes the weight as a function of inputs. These weights are dynamic and play an active part in the prediction. The LPWNN-SNPOM is an improvement of the Wavelet Neural Network with a Hybrid Learning Approach (WNN-HLA). These two models have mainly three differences:i) The LPWNN-SNPOM method contains a bias, ii) The single parameter weights connecting the hidden layer with the output in the WNN-HLA are replaced by polynomial functions of the inputs in the LPWNN-SNPOM, allowing the weights to vary with the changes in the inputs and share the dynamics with the wavelet compartment, iii) Unlike the WNN-HLA, which uses an online optimization approach, the LPWNN-SNPOM makes usage of an offline optimization approach known as the Structured Nonlinear Parameter Optimization Method (SNPOM). The performance and effectiveness of the proposed model are illustrated using several examples, whose results show the feasibility of the proposed model and demonstrate that it improved upon the WNN-HLA and performed better than some other well-known models.Fourthly, the problems of modeling and prediction of time series are investigated with a newly introduced model namely the functional weights wavelet neural network-based state-dependent AR (FWWNN-AR) model. This model consists of combining an enhanced WNN named as a Functional Weights Wavelet Neural Network (FWWNN) to approximate the state dependant coefficients of the AR. The FWWNN-AR model possesses both the advantages of the state-dependent AR model in the description of nonlinear dynamics using few nodes and of the FWWN network in functional approximation considering mutually the time and frequency spaces. The FWWNN is a five-layered network structure comprising of an input layer, wavelet layer, product layer, output layer and functional weight layer. The functional weight layer computes the weight as a function of inputs, making the weights to vary with the inputs and share the dynamics with the wavelet compartment. The proposed model learns the dynamics of nonlinear time series from three distinct levels:AR level, wavelet compartment level and weights level. Different basis functions could be used as functional weight. However, in this thesis, these functions are chosen in such a way that the FWWNN has the standard WNN as one of its component which means that the FWWNN-AR will always perform better than the WNN-AR. Thus the FWWNN-AR can be seen as an improvement of the WNN-AR. Furthermore, the linear version of the FWWNN-AR may be obtained by choosing the translation to tend to infinity. The proposed model is validated by comparing its performances and effectiveness with those achieved by some well-known models on both generated and real nonlinear time series.Finally, the proposed models are investigated for the control issue. To this ends, the FWWNN-AR model, specifically the expanded exponential weight wavelet network based ARX (EW-WNN-ARX) model, which is used as based for developing the internal model of MPC with the aim to control a maritime vehicle. In order to achieve an almost stationary system, the difference of heading angle deviation is used instead of the heading angle. The EW-WNN-ARX model is used to describe the ship motion between the difference of heading angle deviation and the rudder angle of the ship. To represent the ship motion’s nonlinearity, rolling angle is used as model index to allow the model parameters vary with ship moving state. After identification of The EW-WNN-ARX by an offline method known as SNPOM to avoid a possible failure that may occur during online identification procedure, the difference of heading angle deviation are expanded and integrated into mathematical model (MM) to characterize the position tracking error of the ship. The model resulting is a combination of statistical and mathematical model namely the E-EW-WNN-ARX-MM model. The E-EW-WNN-ARX-MM model is then used to develop the state-space type tracking motion model which is used as internal model of the MPC to steer ship moving forward with constant velocity along a predefined reference path. The E-EW-WNN-ARX-MM is developed using data obtained from the Shioji-MARU experimental ship of Tokyo University of Marine Science and Technology of Japan. The modeling and control results achieved demonstrate that the proposed E-EW-WNN-ARX-MM-identified off-line can describe the global nonlinear property of the process over a wide region.