The $k$-sparse parity problem is a classical problem in computational complexity and algorithmic theory, serving as a key benchmark for understanding computational classes. In this paper, we solve the $k$-sparse parity problem with sign stochastic gradient descent, a variant of stochastic gradient descent (SGD) on two-layer fully-connected neural networks. We demonstrate that this approach can efficiently solve the $k$-sparse parity problem on a $d$-dimensional hypercube ($k\leq O(\sqrt{d})$) with a sample complexity of $\tilde{O}(d^{k-1})$ using $2^{\Theta(k)}$ neurons, matching the established $\Omega(d^{k})$ lower bounds of Statistical Query (SQ) models. Our theoretical analysis begins by constructing a good neural network capable of correctly solving the $k$-parity problem. We then demonstrate how a trained neural network with sign SGD can effectively approximate this good network, solving the $k$-parity problem with small statistical errors. To the best of our knowledge, this is the first result that matches the SQ lower bound for solving $k$-sparse parity problem using gradient-based methods.

翻译：$k$-稀疏奇偶校验问题是计算复杂度和算法理论中的一个经典问题，是理解计算复杂类的关键基准。本文利用符号随机梯度下降（随机梯度下降（SGD）的一种变体）在两层全连接神经网络上求解 $k$-稀疏奇偶校验问题。我们证明，该方法能够在 $d$ 维超立方体（$k\leq O(\sqrt{d})$）上，以 $\tilde{O}(d^{k-1})$ 的样本复杂度并使用 $2^{\Theta(k)}$ 个神经元，高效求解 $k$-稀疏奇偶校验问题，从而匹配了统计查询（SQ）模型已知的 $\Omega(d^{k})$ 下界。我们的理论分析首先构建一个能够正确求解 $k$-奇偶校验问题的良好神经网络。随后，我们证明了一个经过符号SGD训练的神经网络如何能够有效逼近该良好网络，并以较小的统计误差求解 $k$-奇偶校验问题。据我们所知，这是首个使用基于梯度的方法求解 $k$-稀疏奇偶校验问题并匹配SQ下界的结果。