Traditional implicit generative models are capable of learning highly complex data distributions. However, their training involves distinguishing real data from synthetically generated data using adversarial discriminators, which can lead to unstable training dynamics and mode dropping issues. In this work, we build on the \textit{invariant statistical loss} (ISL) method introduced in \cite{de2024training}, and extend it to handle heavy-tailed and multivariate data distributions. The data generated by many real-world phenomena can only be properly characterised using heavy-tailed probability distributions, and traditional implicit methods struggle to effectively capture their asymptotic behavior. To address this problem, we introduce a generator trained with ISL, that uses input noise from a generalised Pareto distribution (GPD). We refer to this generative scheme as Pareto-ISL for conciseness. Our experiments demonstrate that Pareto-ISL accurately models the tails of the distributions while still effectively capturing their central characteristics. The original ISL function was conceived for 1D data sets. When the actual data is $n$-dimensional, a straightforward extension of the method was obtained by targeting the $n$ marginal distributions of the data. This approach is computationally infeasible and ineffective in high-dimensional spaces. To overcome this, we extend the 1D approach using random projections and define a new loss function suited for multivariate data, keeping problems tractable by adjusting the number of projections. We assess its performance in multidimensional generative modeling and explore its potential as a pretraining technique for generative adversarial networks (GANs) to prevent mode collapse, reporting promising results and highlighting its robustness across various hyperparameter settings.

翻译：传统的隐式生成模型能够学习高度复杂的数据分布。然而，其训练过程涉及使用对抗判别器区分真实数据与合成生成数据，这可能导致训练动态不稳定及模式丢弃问题。在本工作中，我们基于\cite{de2024training}提出的\textit{不变统计损失}方法，将其扩展至处理重尾与多元数据分布。许多现实世界现象产生的数据仅能通过重尾概率分布进行恰当表征，而传统隐式方法难以有效捕捉其渐近行为。为解决此问题，我们引入了一种采用不变统计损失训练的生成器，该生成器使用来自广义帕累托分布的输入噪声。为简洁起见，我们将此生成方案称为Pareto-ISL。实验表明，Pareto-ISL在准确建模分布尾部的同时，仍能有效捕捉其中心特征。原始不变统计损失函数是为单维数据集设计的。当实际数据为$n$维时，通过针对数据的$n$个边缘分布可获得该方法的直接扩展。此方法在高维空间中计算不可行且效果有限。为克服此限制，我们利用随机投影扩展了单维方法，并定义了适用于多元数据的新损失函数，通过调整投影数量保持问题可处理性。我们评估了其在多维生成建模中的性能，并探索了其作为生成对抗网络预训练技术以防止模式崩溃的潜力，报告了有希望的结果并强调了其在各种超参数设置下的鲁棒性。