We study the complexity of smoothed agnostic learning, recently introduced by~\cite{CKKMS24}, in which the learner competes with the best classifier in a target class under slight Gaussian perturbations of the inputs. Specifically, we focus on the prototypical task of agnostically learning halfspaces under subgaussian distributions in the smoothed model. The best known upper bound for this problem relies on $L_1$-polynomial regression and has complexity $d^{\tilde{O}(1/σ^2) \log(1/ε)}$, where $σ$ is the smoothing parameter and $ε$ is the excess error. Our main result is a Statistical Query (SQ) lower bound providing formal evidence that this upper bound is close to best possible. In more detail, we show that (even for Gaussian marginals) any SQ algorithm for smoothed agnostic learning of halfspaces requires complexity $d^{Ω(1/σ^{2}+\log(1/ε))}$. This is the first non-trivial lower bound on the complexity of this task and nearly matches the known upper bound. Roughly speaking, we show that applying $L_1$-polynomial regression to a smoothed version of the function is essentially best possible. Our techniques involve finding a moment-matching hard distribution by way of linear programming duality. This dual program corresponds exactly to finding a low-degree approximating polynomial to the smoothed version of the target function (which turns out to be the same condition required for the $L_1$-polynomial regression to work). Our explicit SQ lower bound then comes from proving lower bounds on this approximation degree for the class of halfspaces.

翻译：我们研究了由~\cite{CKKMS24}最近提出的平滑不可知学习的计算复杂度，在该框架下，学习器需要在输入受到轻微高斯扰动的情况下，与目标类别中的最佳分类器竞争。具体而言，我们关注平滑模型下，在次高斯分布上不可知学习半空间的典型任务。针对该问题，目前已知的最佳上界依赖于$L_1$多项式回归，其复杂度为$d^{\tilde{O}(1/σ^2) \log(1/ε)}$，其中$σ$为平滑参数，$ε$为超额误差。我们的主要结果是给出了一个统计查询（SQ）下界，为该上界接近最优提供了形式化证据。具体来说，我们证明（即使对于高斯边缘分布）任何用于平滑不可知学习半空间的SQ算法都需要$d^{Ω(1/σ^{2}+\log(1/ε))}$的复杂度。这是针对该任务的第一个非平凡下界，并且几乎匹配已知上界。粗略地说，我们证明了将$L_1$多项式回归应用于函数的平滑版本本质上是接近最优的。我们的技术涉及通过线性规划对偶性找到一个矩匹配的困难分布。该对偶规划恰好对应于为目标函数的平滑版本寻找一个低阶逼近多项式（这恰好是$L_1$多项式回归能够工作的相同条件）。我们通过证明半空间类别的逼近多项式阶数下界，从而得到了显式的SQ下界。