An invertible function is bi-Lipschitz if both the function and its inverse have bounded Lipschitz constants. Nowadays, most Normalizing Flows are bi-Lipschitz by design or by training to limit numerical errors (among other things). In this paper, we discuss the expressivity of bi-Lipschitz Normalizing Flows and identify several target distributions that are difficult to approximate using such models. Then, we characterize the expressivity of bi-Lipschitz Normalizing Flows by giving several lower bounds on the Total Variation distance between these particularly unfavorable distributions and their best possible approximation. Finally, we discuss potential remedies which include using more complex latent distributions.

翻译：一个可逆函数如果其自身及其逆函数都具有有界的Lipschitz常数，则称其为双Lipschitz函数。目前，大多数归一化流在设计中或通过训练方式都采用双Lipschitz约束，以限制数值误差（及其他方面的考虑）。本文探讨了双Lipschitz归一化流的表达能力，并识别出若干难以通过此类模型近似的目标分布。随后，通过给出这些特别不利分布与其最佳可能近似之间的全变差距离下界，刻画了双Lipschitz归一化流的表达能力。最后，我们讨论了潜在改进方案，包括采用更复杂的潜变量分布。