To broaden participation, competitive programming contests may include beginner-level problems that do not require knowledge of advanced Computer Science concepts (e.g., algorithms and data structures). However, since most participants have easy access to AI code-generation tools, these problems often become trivial to solve. For beginner-friendly programming contests that do not prohibit the use of AI tools, we propose Probeable Problems: code writing tasks that provide (1) a problem specification that deliberately omits certain details, and (2) a mechanism to probe for these details by asking clarifying questions and receiving immediate feedback. To evaluate our proposal, we conducted a 2-hour programming contest for undergraduate Computer Science students from multiple institutions, where each student was an active member of their institution's computing club. The contest comprised of six Probeable Problems for which a popular code-generation tool (GitHub Copilot) was unable to generate accurate solutions due to the absence of details. Students were permitted to work individually or in groups, and were free to use AI tools. We obtained consent from 26 groups (67 students) to use their submissions for research. We analyze the extent to which the code submitted by these groups identifies missing details and identify ways in which Probeable Problems can support learning in formal and informal CS educational contexts.
翻译:为扩大参与范围,编程竞赛可纳入无需掌握高级计算机科学概念(如算法与数据结构)的入门级问题。然而,由于多数参赛者能便捷使用AI代码生成工具,此类问题往往变得极易求解。针对不禁止使用AI工具的初学者友好型编程竞赛,我们提出"可探查式问题":这类代码编写任务具备(1)刻意省略特定细节的问题描述,以及(2)通过提出澄清性问题并获取即时反馈来探查这些细节的机制。为评估该方案,我们为来自多所高校的计算机科学本科生举办了为期2小时的编程竞赛,每位参赛者均为所在院校计算机社团的活跃成员。竞赛包含六道可探查式问题,由于细节缺失,主流代码生成工具(GitHub Copilot)无法为此类问题生成准确解决方案。参赛者可以单独或组队参赛,并可自由使用AI工具。我们获得了26组(67名学生)提交作品用于研究的许可。通过分析这些小组提交的代码识别缺失细节的程度,我们揭示了可探查式问题在正式与非正式计算机科学教育场景中支持学习的具体路径。