We study which machine learning algorithms have tight generalization bounds. First, we present conditions that preclude the existence of tight generalization bounds. Specifically, we show that algorithms that have certain inductive biases that cause them to be unstable do not admit tight generalization bounds. Next, we show that algorithms that are sufficiently stable do have tight generalization bounds. We conclude with a simple characterization that relates the existence of tight generalization bounds to the conditional variance of the algorithm's loss.

翻译：我们研究哪些机器学习算法具有紧致的泛化界。首先，我们提出了阻碍紧致泛化界存在的条件。具体而言，我们证明了具有某些导致其不稳定的归纳偏置的算法不允许存在紧致泛化界。接着，我们表明足够稳定的算法确实具有紧致泛化界。最后，我们给出了一个简单的刻画，将紧致泛化界的存在性与算法损失的条 件方差联系起来。