We propose a robust and reliable evaluation metric for generative models by introducing topological and statistical treatments for rigorous support estimation. Existing metrics, such as Inception Score (IS), Frechet Inception Distance (FID), and the variants of Precision and Recall (P&R), heavily rely on supports that are estimated from sample features. However, the reliability of their estimation has not been seriously discussed (and overlooked) even though the quality of the evaluation entirely depends on it. In this paper, we propose Topological Precision and Recall (TopP&R, pronounced 'topper'), which provides a systematic approach to estimating supports, retaining only topologically and statistically important features with a certain level of confidence. This not only makes TopP&R strong for noisy features, but also provides statistical consistency. Our theoretical and experimental results show that TopP&R is robust to outliers and non-independent and identically distributed (Non-IID) perturbations, while accurately capturing the true trend of change in samples. To the best of our knowledge, this is the first evaluation metric focused on the robust estimation of the support and provides its statistical consistency under noise.

翻译：我们提出一种通过引入拓扑与统计处理进行严谨支撑估计的生成模型鲁棒可靠评估指标。现有指标（如Inception Score、Frechet Inception Distance以及各类Precision与Recall变体）严重依赖从样本特征中估计的支撑集，然而尽管评估质量完全取决于支撑估计的可靠性，该可靠性却从未被严肃讨论（且一直被忽视）。本文提出拓扑精度与召回率（TopP&R，发音为'topper'），通过系统性方法估计支撑集，仅保留具有拓扑与统计显著性且达到特定置信水平的特征。这不仅使TopP&R对噪声特征具有强鲁棒性，同时保证了统计一致性。理论与实验结果表明，TopP&R对异常值及非独立同分布（Non-IID）扰动具有鲁棒性，并能准确捕捉样本的真实变化趋势。据我们所知，这是首个聚焦于支撑鲁棒估计并在噪声条件下提供统计一致性的评估指标。