Application Of Data Assimilation And Machine Learning In Estimation Of State And Parameter For Nonlinear Systems

Data assimilation is a method which fuses new observation data with state forecasts from numerical models considering the errors of observation and forecasts.It can solve state and parameter estimation problems for stochastic dynamical systems.Moreover,it has been applied in many areas,like atmospheric science,ocean science,hydrology,natural disasters,global positioning systems,robotics,and computer vision.Machine learning,especially deep learning,can learn complex patterns and nonlinear relationships from data.It can also make predictions based on the past known data.Furthermore,machine learning methods have been used widely in the fields of pattern recognition and image processing,control and optimization,communication,automatic driving and robot simulation.This paper attempts to combine data assimilation and machine learning methods to solve problems of state estimation,parameter estimation and state forecasts for nonlinear stochastic dynamical systems.To solve state estimation problems for nonlinear system,ensemble Kalman filter and particle filter which belong to data assimilation methods have been widely used.In this paper,despite of the two methods,we also investigate hybrid forms of them to get more accurate estimations.The ensemble Kalman filter was proposed by Evensen in 1994.In practice,it can solve state estimation problem for high-dimensional nonlinear systems effectively.But it is optimal when the error distribution is Gaussian in the perspective of maximum posterior probability estimation.Theoretically,particle filter can solve any nonlinear,non-Gaussian problem.When the number of particles is limited,sample degradation problem will occur.As the number of assimilation time steps increases,the weight of the particle gradually concentrates on one particle,while the weights of all other particles are close to zero,then the method loses its effect.There are two main ways to solve the sample degradation problem,one way is to choose an appropriate proposal density function,and the other way is resampling.The accuracy of the state estimation is affected by the choice of proposal density function.We can get more accurate estimations if the proposal density is close to the posterior distribution of state variables.In practice,it is difficult to choose an appropriate proposed density function for nonlinear stochastic systems since we cannot get the accurate posterior distribution.According to resampling,we can select particles based on their weights.Which means that particles with large weights are repeatedly selected,and particles with small weights are abandoned.In some extent,the estimation results can be improved.However,resampling will cause sample impoverishment.In extreme cases,there are only a few different particles after resampling,and the diversity of particles is lost.To solve the problems in state and parameter estimations for nonlinear stochastic systems with ensemble Kalman filter and particle filter,we propose improved methods from different perspectives.To get better forecasts,we use the machine learning method to learn the assimilation process and combine machine learning method and data assimilation method together to do forecasts.The main work of this paper is as follows:(1)We propose to combine unequal weight ensemble Kalman filter and sample regeneration particle filter together to solve state estimation problems for nonlinear systems.At first,we propose unequal weight ensemble Kalman filter to get more suitable proposal density function.Second,to maintain the diversity of particles,we propose sample regenerating particle filter method sampling from other distributions which have the same mean and covariance as the posterior distribution.We combine the two methods to get better state estimation for nonlinear systems.Experiment results show that the presented approach obtains a more accurate forecast than the ensemble Kalman filter and weighted ensemble Kalman filter under Gaussian noise with dense observations.It still performs well in case of sparse observations though more particles are required.Furthermore,for non-Gaussian noise,with an adequate number of particles,the performance of the approach is much better than the ensemble Kalman filter and more robust to noise with nonzero bias.The innovation is as follows:1)In order to choose a suitable proposal density and keep the diversity of the particles,we propose unequal weight ensemble Kalman filter method by calculating the weight of each ensemble member,and propose sample regenerating particle filter method sampling from other distributions which has the same expectation and variance as the posterior distribution.Then,we combine the two methods together to solve state estimations for nonlinear stochastic systems.(2)To solve the problem of state estimation for high-dimensional nonlinear systems with a small size of particles,we proposed optimal combination bootstrap particle filter.At first,it generates N particles from the prior distribution of states.For each element(corresponding to each dimension)in the state variable,it has N possible values.Second,calculate the certainty of each value according to the observation data and sort the N possible values for each element.Third,make a new combination of particles according to the sorting result,then update the weights of particles and resampling.Experiment results show that when dimension is high,compared with ensemble Kalman filter and localization ensemble Kalman filter,it can give best estimation with least computation time.The innovation is as follows:Reorder the state vectors of the particles of a bootstrap particle filter using the associated weight expressed in terms of the likelihood of the measurement error.Then update the weights of particles and resample to get state estimations.We can get particles with large posterior distribution probability and get good state estimations with a small set of particles when the number of dimension is high.(3)To solve joint estimation of state and parameter for nonlinear systems when observation error is large and observation data is sparse,we propose conditionally iterative weighted ensemble transform Kalman filter method.It uses ensemble transformation Kalman filter to define proposal density function to generate analysis ensemble members first.Then it calculates the distance between the background and analysis.If the distance is small,continue the assimilation process.Otherwise,iterate again with ensemble transformation Kalman filter to get analysis ensemble with smaller covariance.Data assimilation can give the optimal state estimation for the dynamical system,but it cannot update the state forecasts without observations.To get more accurate forecasts,we use residual network to learn surrogate model of conditionally iterative weighted ensemble transform Kalman filter method and we use the surrogate model to get better forecasts.The innovations are as follows:1)When observation error is large and observation is sparse,the state and parameter estimation error can propagate through the model over time and accumulate continuously.The model error becomes large and the background error becomes large.Conditionally iterative weighted ensemble transform Kalman filter method try to obtain particles which are more close to the posterior distribution with a more suitable proposal density.Then we can get more accurate estimations of state and parameter.2)We use machine learning methods to learn the relationship between the background and analysis to get the surrogate model for assimilation process.Then try to get more accurate predictions with the surrogate model.