With the significant increase in enrollment in computing-related programs over the past 20 years, lecture sizes have grown correspondingly. In large lectures, instructors face challenges on identifying students' knowledge gaps timely, which is critical for effective teaching. Existing classroom response systems rely on instructor-initiated interactions, which limits their ability to capture the spontaneous knowledge gaps that naturally emerge during lectures. With the widespread adoption of LLMs among students, we recognize these student-AI dialogues as a valuable, student-centered data source for identifying knowledge gaps. In this idea paper, we propose QueryQuilt, a multi-agent LLM framework that automatically detects common knowledge gaps in large-scale lectures by analyzing students' chat logs with AI assistants. QueryQuilt consists of two key components: (1) a Dialogue Agent that responds to student questions while employing probing questions to reveal underlying knowledge gaps, and (2) a Knowledge Gap Identification Agent that systematically analyzes these dialogues to identify knowledge gaps across the student population. By generating frequency distributions of identified gaps, instructors can gain comprehensive insights into class-wide understanding. Our evaluation demonstrates promising results, with QueryQuilt achieving 100% accuracy in identifying knowledge gaps among simulated students and 95% completeness when tested on real student-AI dialogue data. These initial findings indicate the system's potential for facilitate teaching in authentic learning environments. We plan to deploy QueryQuilt in actual classroom settings for comprehensive evaluation, measuring its detection accuracy and impact on instruction.

翻译：过去二十年间，计算相关专业招生人数显著增长，课堂规模相应扩大。在大规模讲座中，教师难以及时识别学生的知识缺口，而这对于有效教学至关重要。现有的课堂应答系统依赖于教师发起的互动，限制了其捕捉讲座过程中自然出现的自发知识缺口的能力。随着大语言模型在学生中的广泛采用，我们认识到这些学生-AI对话是识别知识缺口的宝贵且以学生为中心的数据源。在这篇构想论文中，我们提出QueryQuilt——一个通过分析学生与AI助手的对话日志来自动检测大规模讲座中常见知识缺口的多智能体大语言模型框架。QueryQuilt包含两个关键组件：(1) 对话智能体：在回应学生提问的同时，通过探究性问题揭示潜在知识缺口；(2) 知识缺口识别智能体：系统分析这些对话以识别学生群体中的知识缺口。通过生成已识别缺口的频率分布，教师可以获得对全班理解程度的全面洞察。我们的评估显示出有前景的结果：QueryQuilt在模拟学生中实现100%的知识缺口识别准确率，在真实学生-AI对话数据测试中达到95%的完备性。这些初步发现表明该系统在真实学习环境中辅助教学的潜力。我们计划在实际课堂环境中部署QueryQuilt进行全面评估，测量其检测准确率及对教学的影响。