This work investigates large language models (LLMs) as teachable agents for learning by teaching (LBT). LBT with teachable agents helps learners identify their knowledge gaps and discover new knowledge. However, teachable agents require expensive programming of subject-specific knowledge. While LLMs as teachable agents can reduce the cost, LLMs' over-competence as tutees discourages learners from teaching. We propose a prompting pipeline that restrains LLMs' competence and makes them initiate "why" and "how" questions for effective knowledge-building. We combined these techniques into TeachYou, an LBT environment for algorithm learning, and AlgoBo, an LLM-based tutee chatbot that can simulate misconceptions and unawareness prescribed in its knowledge state. Our technical evaluation confirmed that our prompting pipeline can effectively configure AlgoBo's problem-solving performance. Through a between-subject study with 40 algorithm novices, we also observed that AlgoBo's questions led to knowledge-dense conversations (effect size=0.73). Lastly, we discuss design implications, cost-efficiency, and personalization of LLM-based teachable agents.
翻译:本研究探索将大语言模型作为可教学代理,应用于“通过教学进行学习”的范式。借助可教学代理的“以教促学”方法,能够帮助学习者识别知识盲区并发现新知识。然而,传统可教学代理需要为特定学科知识编写昂贵的程序。虽然采用大语言模型作为可教学代理可降低开发成本,但大语言模型作为学习对象时表现出的过度胜任能力反而会抑制学习者的教学意愿。我们提出了一种提示流水线,通过限制大语言模型的能力水平,引导其生成“为什么”和“如何”类问题以促进有效知识建构。我们将这些技术整合为TeachYou——一个用于算法学习的“以教促学”环境,并开发了AlgoBo——一个能模拟知识状态中预设误解与认知盲区的基于大语言模型的学习者聊天机器人。技术评估证实,我们的提示流水线能有效配置AlgoBo的问题解决表现。通过对40名算法初学者的组间实验,我们观察到AlgoBo提出的问题能引发高知识密度的对话(效应量=0.73)。最后,我们探讨了基于大语言模型的可教学代理的设计启示、成本效益及个性化实现。