Online Modeling Of Nonlinear And Non-stationary Systems Based On Flexible Sparse Models

Based on existing kernel online modeling methods,we systematically study the kernel online learning approaches for non-stationary systems in this paper.Adaptive online modeling of non-linear and non-stationary systems is particularly challenging since the conventional parameter updating may be insufficient to track the changing characteristics with available computational conditions.We adopt the adaptive sparse LIP(linear in the parameter model)models with universal approximation capabilities,which have been intensively studied and widely used due to the availability of many linear-learning algorithms and their model stability and inherent convergence conditions.In recent years,adaptive kernel recursive least squares online algorithms and RBF based structurally optimized LIP online algorithms have been well-studied to achieve better model generalization performance.Kernel recursive least squares algorithm(KRLS),namely kernel adaptive filters,include the mapping procedure which focus on the increasing computational complexity and the proper handling of sparsification of the kernel dictionary,and the weights updating procedure which helps to better track the dynamic characteristics of the changing nonstationarity.We present an improved online kernel recursive least squares algorithm with an inserted forgetting factor,which provides a clearly compatible structure to be improved and helps to better track the dynamic characteristics.The RBF based structurally optimized LIP online models include online optimization of the RBF kernel structure parameters and weight vector.We propose a complete covariance matrix adaptation approach using the CMA-ES algorithms to intermittently optimize the structure parameters,which considers the correlations of input variables and improves models' flexibility.Based on the existing tunable RBF networks,we design a general optimization strategy to update the parameters in the LIP model,which enhances the ability to capture non-stationary dynamics,model instability,and improves the model flexibility andgeneralization ability.A unified optimization strategy of kernel online learning for non-linear and non-stationary systems has been generalized,which improves the applicability of online prediction for non-stationary systems.Experimental results of Lorenz time series demonstrate the effectiveness of the proposed algorithms,and also some properties and problems in the application are explored.