In this paper, we present new high-probability PAC-Bayes bounds for different types of losses. Firstly, for losses with a bounded range, we present a strengthened version of Catoni's bound that holds uniformly for all parameter values. This leads to new fast rate and mixed rate bounds that are interpretable and tighter than previous bounds in the literature. Secondly, for losses with more general tail behaviors, we introduce two new parameter-free bounds: a PAC-Bayes Chernoff analogue when the loss' cumulative generating function is bounded, and a bound when the loss' second moment is bounded. These two bounds are obtained using a new technique based on a discretization of the space of possible events for the "in probability" parameter optimization problem. Finally, we extend all previous results to anytime-valid bounds using a simple technique applicable to any existing bound.

翻译：本文针对不同类型的损失函数提出了新的高概率PAC-Bayes界。首先，对于有界范围损失，我们给出了Catoni界的强化版本，该版本对所有参数值一致成立。这导出了新的快速速率和混合速率界，它们比文献中已有的界更具可解释性且更紧。其次，对于具有更一般尾部行为的损失，我们引入了两个新的无参数界：当损失函数的累积生成函数有界时，给出了PAC-Bayes Chernof界模拟；当损失函数的二阶矩有界时，给出了相应的界。这两个界通过一种基于“依概率”参数优化问题可能事件空间离散化的新技术获得。最后，我们利用一种适用于任何现有界的简单技术，将前述所有结果推广至任意时刻有效界。