$f$-divergences, which quantify discrepancy between probability distributions, are ubiquitous in information theory, machine learning, and statistics. While there are numerous methods for estimating $f$-divergences from data, a limit distribution theory, which quantifies fluctuations of the estimation error, is largely obscure. As limit theorems are pivotal for valid statistical inference, to close this gap, we develop a general methodology for deriving distributional limits for $f$-divergences based on the functional delta method and Hadamard directional differentiability. Focusing on four prominent $f$-divergences -- Kullback-Leibler divergence, $\chi^2$ divergence, squared Hellinger distance, and total variation distance -- we identify sufficient conditions on the population distributions for the existence of distributional limits and characterize the limiting variables. These results are used to derive one- and two-sample limit theorems for Gaussian-smoothed $f$-divergences, both under the null and the alternative. Finally, an application of the limit distribution theory to auditing differential privacy is proposed and analyzed for significance level and power against local alternatives.

翻译：$f$-散度用于量化概率分布之间的差异，在信息论、机器学习和统计学中无处不在。尽管已有多种从数据中估计$f$-散度的方法，但量化估计误差波动的极限分布理论却仍不明确。由于极限定理对于有效的统计推断至关重要，为填补这一空白，我们基于函数delta方法和哈达玛方向可微性，提出了一种推导$f$-散度分布极限的通用方法论。聚焦于四种重要的$f$-散度——Kullback-Leibler散度、$\chi^2$散度、平方Hellinger距离和全变差距离——我们识别了总体分布上存在分布极限的充分条件，并刻画了极限变量。这些结果被用于推导高斯平滑$f$-散度的单样本和双样本极限定理，涵盖原假设和备择假设两种情形。最后，我们提出并分析了极限分布理论在差分隐私审计中的应用，针对显著性水平和局部备择假设下的检验功效进行了评估。