Motivated by challenges in data assimilation for geosciences applications, this work presents theoretical and numerical solutions in nonlinear filtering theory for high dimensional, chaotic, multiple timescale correlated systems. For instance, assimilation of global circulation models in the atmospheric sciences can be of the order 109 degrees of freedom and require assimilation of the order 107 observations during a single day. Due to the high dimensionality, not even the linear data assimilation problem is adequately solved. The models consist of multiple scales spatially and temporally, and are chaotic systems.
The first aspect of this work, develops theoretical results that provides the justification for new filtering algorithms on a lower dimensional problem. This is done in the context of slow-fast dynamical systems where correlation between the slow signal and observation process is allowed. Two results are proved regarding the weak convergence of the nonlinear filtering equation to an averaged filtering equation. In the first scenario, a rate of convergence is achieved using the theory of backward doubly stochastic differential equations to provide a finite dimensional representation of the dual process to the (unnormalized) filtering equation (a linear stochastic partial differential equation). In the second scenario, an intermediate scale forcing is allowed and remarks regarding conditions for correlation between the fast process and observation are stated. The perturb test function approach, where the corrector is the solution of a Poisson equation, is used to prove tightness of the induced probability measures on path space for signed measures as well as the characterization of the limit point.
The second aspect of this work focuses on development of particle filtering algorithms to primarily address the issue of particle collapse in high dimensional chaotic system. First we build on the justification of the theoretical results to produce several particle filtering algorithms that perform estimation on the averaged system, which can result in a dramatic dimensional reduction. We then develop a method for particle filtering in the continuous time signal, discrete time observation case when there is correlation between the signal and observation. An extension to an optimal proposal particle filtering method, which makes use of probabilistic representations for the solution of an optimal control problem, are then given. Lastly, we focus on how to exploit the chaotic properties of the system being filtered to perform assimilation in a lower dimensional subspace. A new approach is developed here using future right singular vectors of a fundamental matrix to construct a projection operator enabling the lower dimensional assimilation. The particle filtering algorithms are tested and compared on standard testbed models from atmospheric science and data assimilation communities: the chaotic Lorenz 1963 and Lorenz 1996 (single and multiple timescale) systems.
