Normalizing flow is a generative modeling approach with efficient sampling. However, Flow-based models suffer two issues: 1) If the target distribution is manifold, due to the unmatch between the dimensions of the latent target distribution and the data distribution, flow-based models might perform badly. 2) Discrete data might make flow-based models collapse into a degenerate mixture of point masses. To sidestep such two issues, we propose PaddingFlow, a novel dequantization method, which improves normalizing flows with padding-dimensional noise. To implement PaddingFlow, only the dimension of normalizing flows needs to be modified. Thus, our method is easy to implement and computationally cheap. Moreover, the padding-dimensional noise is only added to the padding dimension, which means PaddingFlow can dequantize without changing data distributions. Implementing existing dequantization methods needs to change data distributions, which might degrade performance. We validate our method on the main benchmarks of unconditional density estimation, including five tabular datasets and four image datasets for Variational Autoencoder (VAE) models, and the Inverse Kinematics (IK) experiments which are conditional density estimation. The results show that PaddingFlow can perform better in all experiments in this paper, which means PaddingFlow is widely suitable for various tasks. The code is available at: https://github.com/AdamQLMeng/PaddingFlow.

翻译：归一化流是一种高效的生成建模方法。然而，基于流的模型存在两个问题：1）若目标分布为流形，由于潜在目标分布与数据分布之间的维度不匹配，流模型可能表现不佳；2）离散数据可能导致流模型退化为点质量的混合分布。为规避这两个问题，我们提出PaddingFlow——一种新型去量化方法，通过引入填充维度噪声改进归一化流。实现PaddingFlow仅需修改归一化流的维度，因此该方法易于实现且计算成本低廉。此外，填充维度噪声仅添加至填充维度，这意味着PaddingFlow可在不改变数据分布的情况下实现去量化。现有去量化方法需要改变数据分布，可能降低模型性能。我们在无条件密度估计的主要基准测试（包括用于变分自编码器（VAE）模型的五个表格数据集和四个图像数据集）以及条件密度估计的逆运动学（IK）实验中验证了该方法。结果表明，PaddingFlow在本文所有实验中均表现更优，广泛适用于各类任务。代码开源地址：https://github.com/AdamQLMeng/PaddingFlow。