Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and statistics. We propose novel change of measure inequalities via a unified framework based on the data processing inequality, which is surprisingly elementary yet powerful enough to yield novel, tighter inequalities. We provide change of measure inequalities in terms of a broad family of information measures, including $f$-divergences (with Kullback-Leibler divergence and $χ^2$-divergence as special cases), Rényi divergence, and $α$-mutual information (with maximal leakage as a special case). We apply these results to generalization error analysis, PAC-Bayesian theory, differential privacy, and data memorization, obtaining stronger guarantees while recovering best-known results through simplified analyses.

翻译：测度变换不等式将概率测度之间的散度转化为事件概率的显式界，并在学习理论、信息论和统计学中推导概率保证方面发挥重要作用。我们基于数据处理不等式提出了一类新的测度变换不等式，这一框架出人意料地简单，却足以推导出新颖且更紧的不等式。我们给出了以广泛信息测度族表达的测度变换不等式，包括$f$-散度（以KL散度和$\chi^2$散度为特例）、Rényi散度以及$\alpha$-互信息（以最大泄露为特例）。我们将这些结果应用于泛化误差分析、PAC-Bayesian理论、差分隐私和数据记忆化，在通过简化分析恢复已知最优结果的同时，获得了更强的保证。