We present a variety of novel information-theoretic generalization bounds for learning algorithms, from the supersample setting of Steinke & Zakynthinou (2020)-the setting of the "conditional mutual information" framework. Our development exploits projecting the loss pair (obtained from a training instance and a testing instance) down to a single number and correlating loss values with a Rademacher sequence (and its shifted variants). The presented bounds include square-root bounds, fast-rate bounds, including those based on variance and sharpness, and bounds for interpolating algorithms etc. We show theoretically or empirically that these bounds are tighter than all information-theoretic bounds known to date on the same supersample setting.

翻译：我们针对学习算法提出了多种新颖的信息论泛化界，这些界基于Steinke & Zakynthinou（2020）的超样本设置——即"条件互信息"框架。我们的研究工作通过将损失对（由训练实例和测试实例得到）投影为单一数值，并将损失值与Rademacher序列（及其平移变体）相关联来展开。所提出的界包括平方根界、快速率界（包括基于方差和尖锐度的界）、以及用于插值算法的界等。我们通过理论或实证表明，这些界比目前已知的同一超样本设置下所有信息论泛化界都更紧。