We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios. Notably, these bounds address the limitations of existing information-theoretic bounds in the context of stochastic convex optimization (SCO) problems, as explored in the recent work by Haghifam et al. (2023).
翻译:我们通过构建新颖的“邻近假设”矩阵以及引入一类称为“样本条件化假设(SCH)稳定性”的新稳定性概念,提出了新的信息论泛化保证。该方法得到的界在多种学习场景中优于先前信息论界,显著提升了其紧致性。尤其值得注意的是,这些界解决了现有信息论泛化界在随机凸优化(SCO)问题中的局限性——该问题在Haghifam等人(2023)的近期工作中已进行了探讨。