AI code generators like OpenAI Codex have the potential to assist novice programmers by generating code from natural language descriptions, however, over-reliance might negatively impact learning and retention. To explore the implications that AI code generators have on introductory programming, we conducted a controlled experiment with 69 novices (ages 10-17). Learners worked on 45 Python code-authoring tasks, for which half of the learners had access to Codex, each followed by a code-modification task. Our results show that using Codex significantly increased code-authoring performance (1.15x increased completion rate and 1.8x higher scores) while not decreasing performance on manual code-modification tasks. Additionally, learners with access to Codex during the training phase performed slightly better on the evaluation post-tests conducted one week later, although this difference did not reach statistical significance. Of interest, learners with higher Scratch pre-test scores performed significantly better on retention post-tests, if they had prior access to Codex.

翻译：AI代码生成器（如OpenAI Codex）能够根据自然语言描述生成代码，从而为新手程序员提供辅助，但过度依赖可能对学习与知识保留产生负面影响。为探究AI代码生成器对入门编程教学的影响，我们针对69名10-17岁的新手学习者开展了受控实验。要求学习者完成45项Python代码编写任务，其中半数学习者可使用Codex，每项任务后均附有代码修改任务。结果表明：使用Codex显著提升了代码编写表现（完成率提高1.15倍，分数提升1.8倍），同时未降低手动代码修改任务的表现。此外，训练阶段接触Codex的学习者在后续一周后的评估测试中表现略优，但差异未达到统计显著性。值得注意的是，若学习者事先接触过Codex，其Scratch预测试成绩越高，在知识保留后测中的表现显著更优。