Are ensembles really needed for accurate sequential data assimilation!?

Data assimilation estimates the state of dynamical systems from sparse and noisy observations and is used worldwide in numerical weather prediction centers. Accurate data assimilation demands the representation of the time-dependent errors in this state estimate, usually achieved through the propagation of an ensemble of states. Using deep learning, we discover the update step of data assimilation applied to chaotic dynamics. We show that a simple convolutional neural network (CNN) can learn data assimilation, reaching an accuracy as good as that of ensemble-based data assimilation. Crucially, the CNN can achieve this best accuracy with single state forecasts. This is explained by the CNN’s ability to identify local space patterns from this one state, which are used to assess the errors in the analysis. This suggests building a new class of efficient deep learning-based ensemble-free data assimilation algorithms.

The paper is entitled Accurate deep learning-based filtering for chaotic dynamics by identifying instabilities without an ensemble, is published in Chaos, An Interdisciplinary Journal of Nonlinear Science. The full text and its supplementary material are in open access. This effort has been partially funded by the Schmidt Sciences SASIP project.