Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and statistics. We propose novel change of measure inequalities via a unified framework based on the data processing inequality, which is surprisingly elementary yet powerful enough to yield novel, tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We apply these results to generalization error analysis, PAC-Bayesian theory, differential privacy, and data memorization, obtaining stronger guarantees while recovering best-known results through simplified analyses.

翻译：测度变换不等式将概率测度之间的散度转化为事件概率的显式界，在推导学习理论、信息论和统计学中的概率保证方面发挥着重要作用。我们通过一个基于数据处理不等式的统一框架提出了新型测度变换不等式，该方法出人意料地基础却强大到足以产生新颖且更紧的不等式。我们以广泛的信息度量族形式提供了测度变换不等式，包括$f$-散度（Kullback-Leibler散度和$\chi^2$-散度为特例）、Rényi散度以及$\alpha$-互信息（最大泄露为特例）。我们将这些结果应用于泛化误差分析、PAC-贝叶斯理论、差分隐私和数据记忆化，通过简化分析获得了更强的保证并恢复了最佳已知结果。