In this paper, we propose a novel class of change of measure inequalities via a unified framework based on the data processing inequality for $f$-divergences, which is surprisingly elementary yet powerful enough to yield tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We then embed these inequalities into the analysis of generalization error for stochastic learning algorithms, yielding novel and tighter high-probability information-theoretic generalization bounds, while also recovering several best-known results via simplified analyses. A key advantage of our framework is its flexibility: it readily adapts to a range of settings, including the conditional mutual information framework, PAC-Bayesian theory, and differential privacy mechanisms, for which we derive new generalization bounds.

翻译：本文通过基于 $f$ 散度的数据处理不等式的统一框架，提出了一类新颖的测度变换不等式。该框架虽然出人意料地基础，却足够强大以导出更紧的不等式。我们针对一系列广泛的信息测度给出了测度变换不等式，包括 $f$ 散度（以 Kullback-Leibler 散度和 $χ^2$ 散度作为特例）、Rényi 散度以及 $α$-互信息（以最大泄漏作为特例）。随后，我们将这些不等式嵌入到随机学习算法的泛化误差分析中，从而得到了新颖且更紧的高概率信息论泛化界，同时通过简化的分析复现了若干已知的最佳结果。我们框架的一个关键优势在于其灵活性：它能轻松适应多种设置，包括条件互信息框架、PAC-Bayesian 理论以及差分隐私机制，并针对这些设置我们推导出了新的泛化界。