Normalizing Flows explicitly maximize a full-dimensional likelihood on the training data. However, real data is typically only supported on a lower-dimensional manifold leading the model to expend significant compute on modeling noise. Injective Flows fix this by jointly learning a manifold and the distribution on it. So far, they have been limited by restrictive architectures and/or high computational cost. We lift both constraints by a new efficient estimator for the maximum likelihood loss, compatible with free-form bottleneck architectures. We further show that naively learning both the data manifold and the distribution on it can lead to divergent solutions, and use this insight to motivate a stable maximum likelihood training objective. We perform extensive experiments on toy, tabular and image data, demonstrating the competitive performance of the resulting model.

翻译：归一化流通过在训练数据上显式最大化全维似然函数来优化模型。然而，真实数据通常仅支撑于低维流形上，导致模型将大量计算资源用于建模噪声。单射流通过联合学习流形及其上的分布解决了这一问题。此前，此类方法受限于严格的架构约束和/或高计算成本。我们通过提出一种新的最大似然损失高效估计器，同时解除了这两类约束，该估计器兼容自由形式的瓶颈架构。进一步地，我们发现朴素地同时学习数据流形及其上的分布可能导致发散解，并基于此见解推导出稳定的最大似然训练目标。我们在玩具数据集、表格数据及图像数据上进行了广泛实验，验证了所得模型的竞争性性能。