Normalizing Flows have emerged as a powerful brand of generative models, as they not only allow for efficient sampling of complicated target distributions, but also deliver density estimation by construction. We propose here an in-depth comparison of coupling and autoregressive flows, both of the affine and rational quadratic spline type, considering four different architectures: Real-valued Non-Volume Preserving (RealNVP), Masked Autoregressive Flow (MAF), Coupling Rational Quadratic Spline (C-RQS), and Autoregressive Rational Quadratic Spline (A-RQS). We focus on different target distributions of increasing complexity with dimensionality ranging from 4 to 1000. The performances are discussed in terms of different figures of merit: the one-dimensional Wasserstein distance, the one-dimensional Kolmogorov-Smirnov test, the Frobenius norm of the difference between correlation matrices, and the training time. Our results indicate that the A-RQS algorithm stands out both in terms of accuracy and training speed. Nonetheless, all the algorithms are generally able, without much fine-tuning, to learn complex distributions with limited training data and in a reasonable time, of the order of hours on a Tesla V100 GPU. The only exception is the C-RQS, which takes significantly longer to train, and does not always provide good accuracy. All algorithms have been implemented using TensorFlow2 and TensorFlow Probability and made available on GitHub.

翻译：归一化流已成为生成模型的重要分支，它不仅能够高效采样复杂的目标分布，还能通过构造直接提供密度估计。本文对耦合流与自回归流进行了深入比较，涵盖仿射型和有理二次样条型两种类型，并考虑了四种不同架构：实值非体积保持（RealNVP）、掩码自回归流（MAF）、耦合有理二次样条（C-RQS）和自回归有理二次样条（A-RQS）。我们聚焦于复杂度递增、维度从4到1000的不同目标分布。基于多个性能指标讨论结果：一维Wasserstein距离、一维Kolmogorov-Smirnov检验、相关矩阵差异的Frobenius范数以及训练时间。结果表明，A-RQS算法在精度和训练速度方面均表现突出。然而，所有算法在无需大量调参的情况下，通常都能在合理时间内（Tesla V100 GPU上数小时量级）利用有限训练数据学习复杂分布。唯一的例外是C-RQS，其训练时间显著更长，且精度不一定理想。所有算法均基于TensorFlow2和TensorFlow Probability实现，并已在GitHub上开源。