Dialogue plays a crucial role in educational settings, yet existing evaluation methods for educational applications of large language models (LLMs) primarily focus on technical performance or learning outcomes, often neglecting attention to learner-LLM interactions. To narrow this gap, this AIED Doctoral Consortium paper presents an ongoing study employing a dialogue analysis approach to identify effective pedagogical strategies from learner-LLM dialogues. The proposed approach involves dialogue data collection, dialogue act (DA) annotation, DA pattern mining, and predictive model building. Early insights are outlined as an initial step toward future research. The work underscores the need to evaluate LLM-based educational applications by focusing on dialogue dynamics and pedagogical strategies.

翻译：对话在教育场景中扮演着关键角色，然而现有针对大语言模型教育应用的评价方法主要聚焦于技术性能或学习成果，往往忽视了对学习者-大语言模型交互过程的关注。为弥补这一不足，本AIED博士联盟论文提出了一项进行中的研究，采用对话分析方法从学习者-大语言模型对话中识别有效教学策略。所提出的方法包括对话数据收集、对话行为标注、对话模式挖掘以及预测模型构建。本文概述了初步发现，作为未来研究的起点。该工作强调需要通过关注对话动态和教学策略来评估基于大语言模型的教育应用。