Curriculum learning (CL) - training using samples that are generated and presented in a meaningful order - was introduced in the machine learning context around a decade ago. While CL has been extensively used and analysed empirically, there has been very little mathematical justification for its advantages. We introduce a CL model for learning the class of k-parities on d bits of a binary string with a neural network trained by stochastic gradient descent (SGD). We show that a wise choice of training examples, involving two or more product distributions, allows to reduce significantly the computational cost of learning this class of functions, compared to learning under the uniform distribution. We conduct experiments to support our analysis. Furthermore, we show that for another class of functions - namely the `Hamming mixtures' - CL strategies involving a bounded number of product distributions are not beneficial, while we conjecture that CL with unbounded many curriculum steps can learn this class efficiently.

翻译：课程学习（CL）——利用按有意义顺序生成和呈现的样本进行训练——大约十年前被引入机器学习领域。尽管CL已被广泛使用并通过经验分析，但其优势的数学依据却非常有限。我们针对k-奇偶性函数类（在d位二进制串上）提出了一种课程学习模型，该模型采用随机梯度下降（SGD）训练的神经网络进行学习。研究表明，与在均匀分布下学习相比，通过精心选择涉及两种或多种乘积分布的训练样本，可以显著降低学习此类函数的计算成本。我们通过实验支持了上述分析。此外，我们证明对于另一类函数——即"汉明混合"——涉及有界数量乘积分布的CL策略并无益处，但我们推测，采用无界数量课程步骤的CL可以高效学习此类函数。