Data assimilation techniques are crucial for correcting the trajectory when modeling complex physical systems. A recently developed data assimilation method, Latent Ensemble Score Filter (Latent-EnSF), has shown great promise in addressing the key limitation of EnSF for highly sparse observations in high-dimensional and nonlinear data assimilation problems. It performs data assimilation in a latent space for encoded states and observations in every assimilation step, and requires costly full dynamics to be evolved in the original space. In this paper, we introduce Latent Dynamics EnSF (LD-EnSF), a novel methodology that completely avoids the full dynamics evolution and significantly accelerates the data assimilation process, which is especially valuable for complex dynamical problems that require fast data assimilation in real time. To accomplish this, we introduce a novel variant of Latent Dynamics Networks (LDNets) to effectively capture and preserve the system's dynamics within a very low-dimensional latent space. Additionally, we propose a new method for encoding sparse observations into the latent space using Long Short-Term Memory (LSTM) networks, which leverage not only the current step's observations, as in Latent-EnSF, but also all previous steps, thereby improving the accuracy and robustness of the observation encoding. We demonstrate the robustness, accuracy, and efficiency of the proposed method for two challenging dynamical systems with highly sparse (in both space and time) and noisy observations.

翻译：数据同化技术对于在建模复杂物理系统时修正轨迹至关重要。最近开发的一种数据同化方法——潜在集成评分滤波器（Latent-EnSF）——在解决高维非线性数据同化问题中针对高度稀疏观测的EnSF关键局限性方面展现出巨大潜力。该方法在每个同化步骤中，在潜在空间中对编码后的状态和观测进行数据同化，但需要在原始空间中演化计算代价高昂的完整动力学。本文提出潜在动力学EnSF（LD-EnSF），这是一种新颖的方法论，完全避免了完整动力学的演化，并显著加速了数据同化过程，这对于需要实时快速数据同化的复杂动力学问题尤其有价值。为实现这一目标，我们引入了一种新颖的潜在动力学网络（LDNets）变体，以在极低维潜在空间中有效捕获并保持系统的动力学特性。此外，我们提出了一种使用长短期记忆（LSTM）网络将稀疏观测编码到潜在空间的新方法。该方法不仅利用了当前步骤的观测（如Latent-EnSF），还利用了所有先前步骤的观测，从而提高了观测编码的准确性和鲁棒性。我们通过两个具有高度稀疏（在空间和时间上）且含噪声观测的挑战性动力学系统，验证了所提方法的鲁棒性、准确性和效率。