We develop Riemannian approaches to variational autoencoders (VAEs) for PDE-type ambient data with regularizing geometric latent dynamics, which we refer to as VAE-DLM, or VAEs with dynamical latent manifolds. We redevelop the VAE framework such that manifold geometries, subject to our geometric flow, embedded in Euclidean space are learned in the intermediary latent space developed by encoders and decoders. By tailoring the geometric flow in which the latent space evolves, we induce latent geometric properties of our choosing, which are reflected in empirical performance. We reformulate the traditional evidence lower bound (ELBO) loss with a considerate choice of prior. We develop a linear geometric flow with a steady-state regularizing term. This flow requires only automatic differentiation of one time derivative, and can be solved in moderately high dimensions in a physics-informed approach, allowing more expressive latent representations. We discuss how this flow can be formulated as a gradient flow, and maintains entropy away from metric singularity. This, along with an eigenvalue penalization condition, helps ensure the manifold is sufficiently large in measure, nondegenerate, and a canonical geometry, which contribute to a robust representation. Our methods focus on the modified multi-layer perceptron architecture with tanh activations for the manifold encoder-decoder. We demonstrate, on our datasets of interest, our methods perform at least as well as the traditional VAE, and oftentimes better. Our methods can outperform this and a VAE endowed with our proposed architecture, frequently reducing out-of-distribution (OOD) error between 15% to 35% on select datasets. We highlight our method on ambient PDEs whose solutions maintain minimal variation in late times. We provide empirical justification towards how we can improve robust learning for external dynamics with VAEs.

翻译：我们为偏微分方程型环境数据开发了具有正则化几何潜在动态的黎曼变分自编码器(VAE)方法，称之为VAE-DLM（即具有动态潜在流形的VAE）。我们重新构建VAE框架，使得编码器和解码器在中间潜在空间中学习嵌入欧氏空间、受几何流制约的流形几何结构。通过定制潜在空间演化的几何流，我们诱导出所需潜在几何特性，这些特性会在经验性能中得以体现。我们利用审慎选择的先验分布重新表述了传统证据下界(ELBO)损失函数。我们开发了具有稳态正则化项的线性几何流，该流仅需一次时间导数的自动微分，可通过物理信息方法在中高维空间中求解，从而实现更具表达力的潜在表示。我们论证了该流可被表述为梯度流，并在度量奇异性下保持熵值。结合特征值惩罚条件，这有助于确保流形具有足够大的测度、非退化和规范几何结构，从而促进鲁棒表示。我们的方法聚焦于采用双曲正切激活函数的修正多层感知机架构作为流形编解码器。在关注的数据集上，我们验证了该方法性能至少与传统VAE相当，且通常更优。通过赋予所提架构的VAE对比，我们的方法可超越传统VAE，在选定数据集上经常将分布外(OOD)误差降低15%至35%。我们重点展示了该方法在处理后期保持最小变化的含时环境偏微分方程中的效果，并通过实证论证了如何利用VAE改进外部动态的鲁棒学习。