Code language models have emerged as useful tools for various programming tasks, yet they often struggle when it comes to complex ones. In this paper, we explore the potential of curriculum learning in enhancing the performance of these models. While prior research has suggested that curriculum learning does not necessarily help in improving the performance of language models, our results surprisingly show that this may not be the case for code language models. We demonstrate that a well-designed curriculum learning approach significantly improves the accuracy of small decoder-only code language models on the task of code execution, while its effect on code completion is less significant. To explore the potential of curriculum learning, we train multiple GPT models with 1 million parameters each to predict the next token and evaluate them on code completion and execution tasks. Our contributions include proposing a novel code difficulty assessment metric by combining software code measures, investigating the effectiveness of Curriculum Learning for code language models, and introducing a Novel Curriculum Learning schedule that enhances the performance of small decoder-only language models in code execution tasks. The results of this paper open the door for more research on the use of curriculum learning for code language models.

翻译：代码语言模型已成为多种编程任务的有用工具，但在处理复杂任务时往往表现不佳。本文探讨了课程学习在提升此类模型性能方面的潜力。尽管先前研究表明课程学习未必有助于提升语言模型的性能，但我们的结果出人意料地表明，对于代码语言模型而言情况可能并非如此。我们证明，精心设计的课程学习方法能显著提升小型仅解码器代码语言模型在代码执行任务上的准确率，而对代码补全任务的影响则相对有限。为探索课程学习的潜力，我们训练了多个参数量均为100万的GPT模型进行下一词元预测，并在代码补全与执行任务上对其进行了评估。本研究的贡献包括：通过结合软件代码度量指标提出了一种新颖的代码难度评估标准；探究了课程学习在代码语言模型中的有效性；并提出了一种创新的课程学习调度方案，该方案能显著提升小型仅解码器语言模型在代码执行任务中的性能。本文的研究结果为代码语言模型的课程学习应用开辟了更广阔的研究空间。