The massive developments of generative model frameworks and architectures require principled methods for the evaluation of a model's novelty compared to a reference dataset or baseline generative models. While the recent literature has extensively studied the evaluation of the quality, diversity, and generalizability of generative models, the assessment of a model's novelty compared to a baseline model has not been adequately studied in the machine learning community. In this work, we focus on the novelty assessment under multi-modal generative models and attempt to answer the following question: Given the samples of a generative model $\mathcal{G}$ and a reference dataset $\mathcal{S}$, how can we discover and count the modes expressed by $\mathcal{G}$ more frequently than in $\mathcal{S}$. We introduce a spectral approach to the described task and propose the Kernel-based Entropic Novelty (KEN) score to quantify the mode-based novelty of distribution $P_\mathcal{G}$ with respect to distribution $P_\mathcal{S}$. We analytically interpret the behavior of the KEN score under mixture distributions with sub-Gaussian components. Next, we develop a method based on Cholesky decomposition to compute the KEN score from observed samples. We support the KEN-based quantification of novelty by presenting several numerical results on synthetic and real image distributions. Our numerical results indicate the success of the proposed approach in detecting the novel modes and the comparison of state-of-the-art generative models.

翻译：生成模型框架与架构的快速发展需要一套有原则的方法，用于评估模型相较于参考数据集或基线生成模型的新颖性。尽管近期文献已广泛研究了生成模型的质量、多样性和泛化性评估，但机器学习社区对模型相较于基线模型的新颖性评估研究仍不充分。本文聚焦于多模态生成模型下的新颖性评估，试图回答以下问题：给定生成模型 $\mathcal{G}$ 的样本和参考数据集 $\mathcal{S}$，如何发现并统计 $\mathcal{G}$ 比 $\mathcal{S}$ 更频繁表达的模态？我们针对所述任务引入了一种谱方法，并提出了基于核的熵新颖性（KEN）分数，以量化分布 $P_\mathcal{G}$ 相对于分布 $P_\mathcal{S}$ 的模态新颖性。我们从分析角度解释了KEN分数在具有次高斯分量的混合分布下的行为。接着，我们开发了一种基于Cholesky分解的方法，用于从观测样本计算KEN分数。通过在合成和真实图像分布上的多项数值结果，我们验证了基于KEN的新颖性量化方法的有效性。数值结果表明，所提出的方法在检测新模态及比较最新生成模型方面取得了成功。