We propose a robust and reliable evaluation metric for generative models by introducing topological and statistical treatments for rigorous support estimation. Existing metrics, such as Inception Score (IS), Frechet Inception Distance (FID), and the variants of Precision and Recall (P&R), heavily rely on supports that are estimated from sample features. However, the reliability of their estimation has not been seriously discussed (and overlooked) even though the quality of the evaluation entirely depends on it. In this paper, we propose Topological Precision and Recall (TopP&R, pronounced 'topper'), which provides a systematic approach to estimating supports, retaining only topologically and statistically important features with a certain level of confidence. This not only makes TopP&R strong for noisy features, but also provides statistical consistency. Our theoretical and experimental results show that TopP&R is robust to outliers and non-independent and identically distributed (Non-IID) perturbations, while accurately capturing the true trend of change in samples. To the best of our knowledge, this is the first evaluation metric focused on the robust estimation of the support and provides its statistical consistency under noise.

翻译：我们提出了一种鲁棒且可靠的生成模型评估指标，通过引入拓扑与统计方法实现严格的支撑估计。现有指标（如Inception Score、Frechet Inception Distance以及各类Precision and Recall变体）严重依赖从样本特征中估计出的支撑集，然而尽管评估质量完全取决于支撑估计的可靠性，其可信度至今未被充分探讨（甚至被忽视）。本文提出拓扑精确率与召回率（TopP&R，读作'topper'），提供了一种系统的支撑估计方法，仅保留具有统计显著性与拓扑重要性的特征，并赋予其置信水平。这不仅使TopP&R对噪声特征具有强鲁棒性，还提供了统计一致性。理论与实验表明，TopP&R对异常值与非独立同分布扰动均表现稳健，同时能准确捕捉样本变化的真实趋势。据我们所知，这是首个聚焦于支撑鲁棒估计的评估指标，并在噪声条件下证明了其统计一致性。