Deep generative models have demonstrated the ability to generate complex, high-dimensional, and photo-realistic data. However, a unified framework for evaluating different generative modeling families remains a challenge. Indeed, likelihood-based metrics do not apply in many cases while pure sample-based metrics such as FID fail to capture known failure modes such as overfitting on training data. In this work, we introduce the Feature Likelihood Score (FLS), a parametric sample-based score that uses density estimation to quantitatively measure the quality/diversity of generated samples while taking into account overfitting. We empirically demonstrate the ability of FLS to identify specific overfitting problem cases, even when previously proposed metrics fail. We further perform an extensive experimental evaluation on various image datasets and model classes. Our results indicate that FLS matches intuitions of previous metrics, such as FID, while providing a more holistic evaluation of generative models that highlights models whose generalization abilities are under or overappreciated. Code for computing FLS is provided at https://github.com/marcojira/fls

翻译：深度生成模型已展现出生成复杂、高维且逼真数据的能力。然而，针对不同生成模型家族的统一评估框架仍是一个挑战。具体而言，基于似然的指标在许多场景下不适用，而诸如FID等纯粹基于样本的指标则无法捕捉已知的失败模式（如对训练数据的过拟合）。本研究提出特征似然分数（FLS）——一种基于参数化样本的评估方法，通过密度估计定量衡量生成样本的质量/多样性，同时考虑过拟合风险。我们通过实验证明，即使在已有指标失效的情况下，FLS仍能有效识别特定过拟合问题案例。进一步在多种图像数据集和模型类别上开展广泛实验评估，结果表明：FLS在匹配FID等传统指标直觉判断的同时，能够对生成模型进行更全面的评估，揭示其泛化能力被低估或高估的模型。FLS计算代码已开源：https://github.com/marcojira/fls