We study the problem of learning mixtures of linear classifiers under Gaussian covariates. Given sample access to a mixture of $r$ distributions on $\mathbb{R}^n$ of the form $(\mathbf{x},y_{\ell})$, $\ell\in [r]$, where $\mathbf{x}\sim\mathcal{N}(0,\mathbf{I}_n)$ and $y_\ell=\mathrm{sign}(\langle\mathbf{v}_\ell,\mathbf{x}\rangle)$ for an unknown unit vector $\mathbf{v}_\ell$, the goal is to learn the underlying distribution in total variation distance. Our main result is a Statistical Query (SQ) lower bound suggesting that known algorithms for this problem are essentially best possible, even for the special case of uniform mixtures. In particular, we show that the complexity of any SQ algorithm for the problem is $n^{\mathrm{poly}(1/\Delta) \log(r)}$, where $\Delta$ is a lower bound on the pairwise $\ell_2$-separation between the $\mathbf{v}_\ell$'s. The key technical ingredient underlying our result is a new construction of spherical designs that may be of independent interest.
翻译:我们研究高斯协变量下线性分类器混合的学习问题。假设可从混合分布中获取样本,该混合由$r$个形如$(\mathbf{x},y_{\ell})$($\ell\in [r]$)的$\mathbb{R}^n$上的子分布组成,其中$\mathbf{x}\sim\mathcal{N}(0,\mathbf{I}_n)$,且对于未知单位向量$\mathbf{v}_\ell$有$y_\ell=\mathrm{sign}(\langle\mathbf{v}_\ell,\mathbf{x}\rangle)$。目标是学习总变差距离下的潜在分布。我们的主要结果是统计查询(SQ)下界,表明该问题的已知算法本质上是可能的最优解,即使在均匀混合的特殊情形下也是如此。特别地,我们证明该问题的任意SQ算法的复杂度为$n^{\mathrm{poly}(1/\Delta) \log(r)}$,其中$\Delta$是各$\mathbf{v}_\ell$之间逐对$\ell_2$距离的下界。支撑这一结果的关键技术要素是球面设计的新构造,该构造本身可能具有独立研究价值。