Curriculum Learning (CL) aims to improve the outcome of model training by estimating the difficulty of samples and scheduling them accordingly. In NLP, difficulty is commonly approximated using task-agnostic linguistic heuristics or human intuition, implicitly assuming that these signals correlate with what neural models find difficult to learn. We propose a four-quadrant categorisation of difficulty signals -- human vs. model and task-agnostic vs. task-dependent -- and systematically analyse their interactions on a natural language understanding dataset. We find that task-agnostic features behave largely independently and that only task-dependent features align. These findings challenge common CL intuitions and highlight the need for lightweight, task-dependent difficulty estimators that better reflect model learning behaviour.

翻译：课程学习旨在通过估计样本难度并据此安排训练顺序来改进模型训练效果。在自然语言处理领域，难度通常通过任务无关的语言学启发式方法或人类直觉进行近似估计，这隐含地假设这些信号与神经网络模型的学习难点相关。我们提出了难度信号的四种分类象限——人类评估与模型评估、任务无关与任务相关，并在自然语言理解数据集上系统分析了它们的相互作用。研究发现：任务无关特征基本保持独立行为，只有任务相关特征呈现一致性。这些发现挑战了课程学习的常见直觉认知，并凸显了开发轻量级、任务相关的难度估计器的必要性，以更准确地反映模型的学习行为。