AI-powered programming assistants are increasingly gaining popularity, with GitHub Copilot alone used by over a million developers worldwide. These tools are far from perfect, however, producing code suggestions that may be incorrect in subtle ways. As a result, developers face a new challenge: validating AI's suggestions. This paper explores whether Live Programming (LP), a continuous display of a program's runtime values, can help address this challenge. To answer this question, we built a Python editor that combines an AI-powered programming assistant with an existing LP environment. Using this environment in a between-subjects study (N=17), we found that by lowering the cost of validation by execution, LP can mitigate over- and under-reliance on AI-generated programs and reduce the cognitive load of validation for certain types of tasks.
翻译:AI驱动的编程助手正日益普及,仅GitHub Copilot就已服务全球超过百万开发者。然而这些工具远非完美,其生成的代码建议可能存在不易察觉的错误,导致开发者面临全新挑战:如何验证AI的生成结果。本文探索实时编程(Live Programming, LP)技术——即持续显示程序运行时值的编程范式——能否应对这一挑战。为解答该问题,我们构建了一个集成AI编程助手与现有LP环境的Python编辑器。通过组间实验(N=17)发现,LP通过降低执行验证的成本,能够缓解对AI生成程序的过度依赖与不足依赖,并降低特定类型任务的验证认知负荷。