Normalizing Flows explicitly maximize a full-dimensional likelihood on the training data. However, real data is typically only supported on a lower-dimensional manifold leading the model to expend significant compute on modeling noise. Injective Flows fix this by jointly learning a manifold and the distribution on it. So far, they have been limited by restrictive architectures and/or high computational cost. We lift both constraints by a new efficient estimator for the maximum likelihood loss, compatible with free-form bottleneck architectures. We further show that naively learning both the data manifold and the distribution on it can lead to divergent solutions, and use this insight to motivate a stable maximum likelihood training objective. We perform extensive experiments on toy, tabular and image data, demonstrating the competitive performance of the resulting model.

翻译：归一化流（Normalizing Flows）显式地在训练数据上最大化全维度的似然。然而，真实数据通常仅存在于一个低维流形上，这导致模型需要耗费大量计算资源来建模噪声。可逆流通过联合学习一个流形及其上的分布来解决此问题。迄今为止，它们一直受到限制性架构和/或高计算成本的制约。我们通过一种新的、兼容自由形式瓶颈架构的最大似然损失高效估计器，同时解除了这两项限制。我们进一步表明，简单地同时学习数据流形及其上的分布可能导致发散的解决方案，并利用这一见解来提出一种稳定的最大似然训练目标。我们在玩具数据、表格数据和图像数据上进行了广泛的实验，证明了所提出模型具有竞争力的性能。