How can AI enhance creative coding experiences for families? This study explores the potential of large language models (LLMs) in helping families with creative coding using Scratch. Based on our previous user study involving a prototype AI assistant, we devised three evaluation scenarios to determine if LLMs could help families comprehend game code, debug programs, and generate new ideas for future projects. We utilized 22 Scratch projects for each scenario and generated responses from LLMs with and without practice tasks, resulting in 120 creative coding support scenario datasets. In addition, the authors independently evaluated their precision, pedagogical value, and age-appropriate language. Our findings show that LLMs achieved an overall success rate of more than 80\% on the different tasks and evaluation criteria. This research offers valuable information on using LLMs for creative family coding and presents design guidelines for future AI-supported coding applications. Our evaluation framework, together with our labeled evaluation data, is publicly available.

翻译：人工智能如何增强家庭的创意编程体验？本研究探索了大语言模型（LLMs）在辅助家庭使用Scratch进行创意编程的潜力。基于先前包含原型AI助手的用户研究，我们设计了三种评估场景，以检验LLMs能否协助用户理解游戏代码、调试程序以及为未来项目生成新创意。每个场景采用22个Scratch项目，分别获取含练习任务与不含练习任务的LLMs生成响应，共计构建120个创意编程支持场景数据集。此外，研究者独立评估了数据的精准度、教学价值及适龄语言水平。研究结果显示，LLMs在不同任务与评估标准下的整体成功率超过80%。本研究为将LLMs应用于家庭创意编程提供了有价值的参考，并为未来AI支持的编程应用提出了设计指南。我们的评估框架及标注评估数据已公开提供。