In this paper, we propose a novel class of change of measure inequalities via a unified framework based on the data processing inequality for $f$-divergences, which is surprisingly elementary yet powerful enough to yield tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We then embed these inequalities into the analysis of generalization error for stochastic learning algorithms, yielding novel and tighter high-probability information-theoretic generalization bounds, while also recovering several best-known results via simplified analyses. A key advantage of our framework is its flexibility: it readily adapts to a range of settings, including the conditional mutual information framework, PAC-Bayesian theory, and differential privacy mechanisms, for which we derive new generalization bounds.

翻译：本文通过基于$f$散度的数据处理不等式的统一框架，提出了一类新颖的测度变换不等式。该框架虽然出人意料地基础，却足够强大以导出更紧的不等式。我们以广泛的信息测度族（包括$f$散度——以Kullback-Leibler散度和$χ^2$散度为特例、Rényi散度以及$α$互信息——以最大泄漏为特例）为工具，给出了测度变换不等式。随后，我们将这些不等式嵌入到随机学习算法泛化误差的分析中，从而得到新颖且更紧的高概率信息论泛化界，同时通过简化的分析恢复了若干已知的最佳结果。我们框架的一个关键优势在于其灵活性：它能轻松适应多种设定，包括条件互信息框架、PAC-Bayesian理论以及差分隐私机制，并为此我们推导出了新的泛化界。