We demonstrate a compactness result holding broadly across supervised learning with a general class of loss functions: Any hypothesis class $H$ is learnable with transductive sample complexity $m$ precisely when all of its finite projections are learnable with sample complexity $m$. We prove that this exact form of compactness holds for realizable and agnostic learning with respect to any proper metric loss function (e.g., any norm on $\mathbb{R}^d$) and any continuous loss on a compact space (e.g., cross-entropy, squared loss). For realizable learning with improper metric losses, we show that exact compactness of sample complexity can fail, and provide matching upper and lower bounds of a factor of 2 on the extent to which such sample complexities can differ. We conjecture that larger gaps are possible for the agnostic case. Furthermore, invoking the equivalence between sample complexities in the PAC and transductive models (up to lower order factors, in the realizable case) permits us to directly port our results to the PAC model, revealing an almost-exact form of compactness holding broadly in PAC learning.

翻译：我们证明了在监督学习中，对于一类广泛的损失函数普遍存在一个紧致性结果：任何假设类$H$具有归纳样本复杂度$m$可学习，当且仅当其所有有限投影都具有样本复杂度$m$可学习。我们证明这种精确的紧致性形式对于任何真度量损失函数（例如$\mathbb{R}^d$上的任意范数）和紧空间上的任意连续损失（例如交叉熵、平方损失）的可实现学习与不可知学习均成立。对于使用非真度量损失的可实现学习，我们证明了样本复杂度的精确紧致性可能失效，并给出了此类样本复杂度差异可达2倍因子的匹配上下界。我们推测在不可知情形下可能存在更大的差距。此外，通过调用PAC模型与归纳模型中样本复杂度的等价性（在可实现情形下相差低阶因子），我们可以直接将结果移植到PAC模型，从而揭示在PAC学习中广泛成立的几乎精确的紧致性形式。