While PAC-Bayes is now an established learning framework for light-tailed losses (\emph{e.g.}, subgaussian or subexponential), its extension to the case of heavy-tailed losses remains largely uncharted and has attracted a growing interest in recent years. We contribute PAC-Bayes generalisation bounds for heavy-tailed losses under the sole assumption of bounded variance of the loss function. Under that assumption, we extend previous results from \citet{kuzborskij2019efron}. Our key technical contribution is exploiting an extention of Markov's inequality for supermartingales. Our proof technique unifies and extends different PAC-Bayesian frameworks by providing bounds for unbounded martingales as well as bounds for batch and online learning with heavy-tailed losses.

翻译：虽然PAC-Bayes现已是一种针对轻尾损失（例如，亚高斯或亚指数）的成熟学习框架，但其在重尾损失情况下的扩展仍 largely 未被探索，且近年来吸引了日益增长的研究兴趣。我们在仅假设损失函数方差有界的条件下，为重尾损失推导了PAC-Bayes泛化界。在该假设下，我们扩展了Kuzborskij等人（2019年）的先前结果。我们的关键技术贡献在于利用超鞅的马尔可夫不等式扩展形式。通过为无界鞅以及重尾损失的批量和在线学习提供界，我们的证明技术统一并扩展了不同的PAC-Bayes框架。