We study the task of agnostic learning of multiclass linear classifiers under the Gaussian distribution. Given labeled examples $(x, y)$ from a distribution over $\mathbb{R}^d \times [k]$, with Gaussian $x$-marginal, the goal is to output a hypothesis whose error is comparable to that of the best $k$-class linear classifier. While the binary case $k=2$ has a well-developed algorithmic theory, much less is known for $k \ge 3$. Even for $k=3$, prior robust algorithms incur exponential dependence on the inverse of the desired accuracy in both complexity and representation size. In this work, we develop new structural results for multiclass linear classifiers and use them to design fully polynomial-time robust learners with dimension-independent error guarantees. Our first result shows that the standard multiclass perceptron algorithm requires super-polynomially many samples and updates, even with clean labels and Gaussian marginals, revealing a basic obstruction absent in the binary case. Our main positive result is a pairwise improper-learning framework which yields an efficient learner with error $\widetilde O(k^{3/2}\sqrt{\mathrm{opt}})+ε$ for general $k$. Additionally, we develop a sharper localization-based framework which leads to error $O(\mathrm{opt})+ε$ for $k=3$, and error $\mathrm{poly}(k)\mathrm{opt}+ε$ for geometrically regular $k$-class linear classifiers.

翻译：我们研究了在高斯分布下进行多类线性分类器的不可知学习任务。给定来自 $\mathbb{R}^d \times [k]$ 分布上的带标签样本 $(x, y)$（其中 $x$ 的边缘分布为高斯分布），目标是输出一个假设，其误差与最优的 $k$ 类线性分类器相当。虽然二元情形 $k=2$ 具有完善的算法理论，但对于 $k \ge 3$ 的情况则知之甚少。即使对于 $k=3$，先前的鲁棒算法在复杂度和表示规模上均依赖于所需精度的指数的逆。本文中，我们开发了多类线性分类器的新结构性结果，并利用它们设计出具有维度无关误差保证的完全多项式时间鲁棒学习器。我们的第一个结果表明，即使标签纯净且边缘分布为高斯分布，标准的多类感知机算法也需要超多项式规模的样本和更新，揭示了二元情形中不存在的基本障碍。我们主要的正面结果是一个两两非恰当学习框架，该框架为一般 $k$ 情形产生一个高效学习器，其误差为 $\widetilde O(k^{3/2}\sqrt{\mathrm{opt}})+ε$。此外，我们开发了一个更尖锐的基于局部化的框架，该框架在 $k=3$ 时达到误差 $O(\mathrm{opt})+ε$，在几何正则的 $k$ 类线性分类器上达到误差 $\mathrm{poly}(k)\mathrm{opt}+ε$。