We develop Riemannian approaches to variational autoencoders (VAEs) for PDE-type ambient data with regularizing geometric latent dynamics, which we refer to as VAE-DLM, or VAEs with dynamical latent manifolds. We redevelop the VAE framework such that manifold geometries, subject to our geometric flow, embedded in Euclidean space are learned in the intermediary latent space developed by encoders and decoders. By tailoring the geometric flow in which the latent space evolves, we induce latent geometric properties of our choosing, which are reflected in empirical performance. We reformulate the traditional evidence lower bound (ELBO) loss with a considerate choice of prior. We develop a linear geometric flow with a steady-state regularizing term. This flow requires only automatic differentiation of one time derivative, and can be solved in moderately high dimensions in a physics-informed approach, allowing more expressive latent representations. We discuss how this flow can be formulated as a gradient flow, and maintains entropy away from metric singularity. This, along with an eigenvalue penalization condition, helps ensure the manifold is sufficiently large in measure, nondegenerate, and a canonical geometry, which contribute to a robust representation. Our methods focus on the modified multi-layer perceptron architecture with tanh activations for the manifold encoder-decoder. We demonstrate, on our datasets of interest, our methods perform at least as well as the traditional VAE, and oftentimes better. Our methods can outperform this and a VAE endowed with our proposed architecture by up to 25% reduction in out-of-distribution (OOD) error and potentially greater. We highlight our method on ambient PDEs whose solutions maintain minimal variation in late times. We provide empirical justification towards how we can improve robust learning for external dynamics with VAEs.

翻译：我们针对具有正则化几何潜在动力学的偏微分方程型环境数据，发展了黎曼几何方法下的变分自编码器（VAE），我们将其称为VAE-DLM，即具有动态潜在流形的VAE。我们重新构建了VAE框架，使得在编码器与解码器构建的中间潜在空间中，能够学习嵌入欧几里得空间且受我们几何流约束的流形几何结构。通过定制潜在空间演化所遵循的几何流，我们能够引入所期望的潜在几何特性，这些特性在实证性能中得以体现。我们以审慎的先验选择重新构建了传统的证据下界（ELBO）损失函数。我们发展了一种带有稳态正则项的线性几何流。该流仅需对一阶时间导数进行自动微分，并可在物理信息驱动的方法中于中等高维度下求解，从而获得更具表达力的潜在表示。我们讨论了如何将该流表述为梯度流，并使其在度量非奇异性条件下保持熵。这一点，结合特征值惩罚条件，有助于确保流形在测度上充分大、非退化且具有规范几何结构，从而促进鲁棒表示的形成。我们的方法聚焦于采用tanh激活函数的改进型多层感知机架构，用于流形编码器-解码器。我们在感兴趣的数据集上证明，我们的方法性能至少与传统VAE相当，且通常更优。相较于传统VAE及采用我们提出架构的VAE，我们的方法能将分布外（OOD）误差降低高达25%，甚至可能更多。我们重点展示了该方法在环境偏微分方程上的应用，这些方程的解在后期保持最小变化。我们提供了实证依据，说明如何利用VAE提升对外部动力学的鲁棒学习能力。