We study the supervised learning paradigm called Learning Using Privileged Information, first suggested by Vapnik and Vashist (2009). In this paradigm, in addition to the examples and labels, additional (privileged) information is provided only for training examples. The goal is to use this information to improve the classification accuracy of the resulting classifier, where this classifier can only use the non-privileged information of new example instances to predict their label. We study the theory of privileged learning with the zero-one loss under the natural Privileged ERM algorithm proposed in Pechyony and Vapnik (2010a). We provide a counter example to a claim made in that work regarding the VC dimension of the loss class induced by this problem; We conclude that the claim is incorrect. We then provide a correct VC dimension analysis which gives both lower and upper bounds on the capacity of the Privileged ERM loss class. We further show, via a generalization analysis, that worst-case guarantees for Privileged ERM cannot improve over standard non-privileged ERM, unless the capacity of the privileged information is similar or smaller to that of the non-privileged information. This result points to an important limitation of the Privileged ERM approach. In our closing discussion, we suggest another way in which Privileged ERM might still be helpful, even when the capacity of the privileged information is large.

翻译：我们研究了一种名为“利用特权信息学习”的监督学习范式，该范式最早由Vapnik和Vashist (2009)提出。在该范式中，除了样本和标签外，训练样本还额外提供了（特权）信息。目标是利用这些信息提高所得分类器的分类准确率，而该分类器仅能利用新样本的非特权信息来预测其标签。我们研究了在Pechyony和Vapnik (2010a)提出的自然特权经验风险最小化算法下，采用零一损失函数的特权学习理论。我们针对该工作中关于由此问题产生的损失类VC维的一个结论给出了反例；因此认为该结论不正确。随后，我们给出了正确的VC维分析，该分析提供了特权经验风险最小化损失类容量的下界和上界。通过泛化分析，我们进一步证明，除非特权信息的容量与非特权信息的容量相近或更小，否则特权经验风险最小化的最坏情况保证无法优于标准的非特权经验风险最小化。这一结果指出了特权经验风险最小化方法的一个重要局限性。在最后的讨论中，我们提出了即使在特权信息容量较大时，特权经验风险最小化仍可能发挥作用的另一种方式。