Current PAC-Bayes generalisation bounds are restricted to scalar metrics of performance, such as the loss or error rate. However, one ideally wants more information-rich certificates that control the entire distribution of possible outcomes, such as the distribution of the test loss in regression, or the probabilities of different mis-classifications. We provide the first PAC-Bayes bound capable of providing such rich information by bounding the Kullback-Leibler divergence between the empirical and true probabilities of a set of $M$ error types, which can either be discretized loss values for regression, or the elements of the confusion matrix (or a partition thereof) for classification. We transform our bound into a differentiable training objective. Our bound is especially useful in cases where the severity of different mis-classifications may change over time; existing PAC-Bayes bounds can only bound a particular pre-decided weighting of the error types. In contrast our bound implicitly controls all uncountably many weightings simultaneously.

翻译：现有的PAC-Bayes泛化界仅限于性能的标量度量，如损失或错误率。然而，理想情况下需要信息更丰富的证书来控制所有可能结果的完整分布，例如回归中测试损失的分布，或不同错误分类的概率。我们通过约束$M$种误差类型的经验概率与真实概率之间的Kullback-Leibler散度，提供了首个能够提供此类丰富信息的PAC-Bayes界——这些误差类型可以是回归中离散化的损失值，也可以是分类任务中混淆矩阵（或其划分）的元素。我们将该界转化为可微分的训练目标。该界在以下场景中尤为有用：不同错误分类的严重性可能随时间变化；而现有的PAC-Bayes界只能约束预先设定的误差类型加权组合。相比之下，我们的界能同时隐式控制所有不可数无穷多个加权组合。