We present a new general-purpose algorithm for learning classes of $[0,1]$-valued functions in a generalization of the prediction model, and prove a general upper bound on the expected absolute error of this algorithm in terms of a scale-sensitive generalization of the Vapnik dimension proposed by Alon, Ben-David, Cesa-Bianchi and Haussler. We give lower bounds implying that our upper bounds cannot be improved by more than a constant factor in general. We apply this result, together with techniques due to Haussler and to Benedek and Itai, to obtain new upper bounds on packing numbers in terms of this scale-sensitive notion of dimension. Using a different technique, we obtain new bounds on packing numbers in terms of Kearns and Schapire's fat-shattering function. We show how to apply both packing bounds to obtain improved general bounds on the sample complexity of agnostic learning. For each $\epsilon > 0$, we establish weaker sufficient and stronger necessary conditions for a class of $[0,1]$-valued functions to be agnostically learnable to within $\epsilon$, and to be an $\epsilon$-uniform Glivenko-Cantelli class. This is a manuscript that was accepted by JCSS, together with a correction.
翻译:我们提出了一种适用于$[0,1]$值函数类学习的通用新算法,该算法基于预测模型的推广形式,并证明了该算法期望绝对误差的通用上界,该上界由Alon、Ben-David、Cesa-Bianchi和Haussler提出的Vapnik维数的尺度敏感推广所刻画。我们给出了下界,表明在一般情况下,我们的上界至多只能通过一个常数因子进行改进。结合Haussler以及Benedek与Itai的技术,我们将该结果应用于获得该尺度敏感维度概念下packing数的新上界。利用另一种技术,我们得到了关于Kearns与Schapire的fat-shattering函数的packing数新上界。我们展示了如何将这两个packing界应用于改进agnostic学习样本复杂度的通用界。对于每个$\epsilon > 0$,我们为$[0,1]$值函数类建立了在$\epsilon$范围内进行agnostic学习的充分条件(较弱的版本)与必要条件(较强的版本),并证明了该类为$\epsilon$-一致Glivenko-Cantelli类的充要条件。本文为已被JCSS接收的稿件(附有勘误)。