We propose a robust and reliable evaluation metric for generative models by introducing topological and statistical treatments for rigorous support estimation. Existing metrics, such as Inception Score (IS), Frechet Inception Distance (FID), and the variants of Precision and Recall (P&R), heavily rely on supports that are estimated from sample features. However, the reliability of their estimation has not been seriously discussed (and overlooked) even though the quality of the evaluation entirely depends on it. In this paper, we propose Topological Precision and Recall (TopP&R, pronounced 'topper'), which provides a systematic approach to estimating supports, retaining only topologically and statistically important features with a certain level of confidence. This not only makes TopP&R strong for noisy features, but also provides statistical consistency. Our theoretical and experimental results show that TopP&R is robust to outliers and non-independent and identically distributed (Non-IID) perturbations, while accurately capturing the true trend of change in samples. To the best of our knowledge, this is the first evaluation metric focused on the robust estimation of the support and provides its statistical consistency under noise.

翻译：我们提出一种通过引入拓扑与统计处理进行严格支撑估计的生成模型鲁棒可靠评估指标。现有指标（如Inception Score (IS)、Frechet Inception Distance (FID)及召回率与精确率变体）严重依赖于从样本特征中估计的支撑集。然而，尽管评估质量完全取决于支撑估计的可靠性，其估计可靠性却未得到认真探讨（甚至被忽视）。本文提出拓扑精确率与召回率（TopP&R，读作"topper"），该指标提供系统性支撑估计方法，仅保留具有拓扑与统计重要性且达到一定置信水平的特征。这不仅使TopP&R对噪声特征具有强鲁棒性，还保证了统计一致性。理论分析与实验结果表明，TopP&R对异常值与独立同分布（Non-IID）扰动具有鲁棒性，同时能准确捕捉样本的真实变化趋势。据我们所知，这是首个聚焦于支撑鲁棒估计并能在噪声条件下保持统计一致性的评估指标。