We demonstrate a compactness result holding broadly across supervised learning with a general class of loss functions: Any hypothesis class $H$ is learnable with transductive sample complexity $m$ precisely when all of its finite projections are learnable with sample complexity $m$. We prove that this exact form of compactness holds for realizable and agnostic learning with respect to any proper metric loss function (e.g., any norm on $\mathbb{R}^d$) and any continuous loss on a compact space (e.g., cross-entropy, squared loss). For realizable learning with improper metric losses, we show that exact compactness of sample complexity can fail, and provide matching upper and lower bounds of a factor of 2 on the extent to which such sample complexities can differ. We conjecture that larger gaps are possible for the agnostic case. Furthermore, invoking the equivalence between sample complexities in the PAC and transductive models (up to lower order factors, in the realizable case) permits us to directly port our results to the PAC model, revealing an almost-exact form of compactness holding broadly in PAC learning.

翻译：我们证明了一个在监督学习中广泛成立的紧致性结果，适用于一大类损失函数：任何假设类 $H$ 在具有归纳样本复杂度 $m$ 时可学习，当且仅当其所有有限投影在样本复杂度 $m$ 下可学习。我们证明了这种精确的紧致性形式对于任何真度量损失函数（例如 $\mathbb{R}^d$ 上的任何范数）和任何紧空间上的连续损失（例如交叉熵、平方损失）下的可实现学习和不可知学习均成立。对于使用非真度量损失的可实现学习，我们证明了样本复杂度的精确紧致性可能不成立，并给出了此类样本复杂度差异可达 2 倍的匹配上下界。我们推测在不可知情况下可能存在更大的差距。此外，通过利用 PAC 模型与归纳模型中样本复杂度的等价性（在可实现情况下，忽略低阶因子），我们可以直接将我们的结果移植到 PAC 模型，从而揭示在 PAC 学习中广泛成立的几乎精确的紧致性形式。