We present an evaluation framework for interactive dialogue assessment in the context of English as a Second Language (ESL) speakers. Our framework collects dialogue-level interactivity labels (e.g., topic management; 4 labels in total) and micro-level span features (e.g., backchannels; 17 features in total). Given our annotated data, we study how the micro-level features influence the (higher level) interactivity quality of ESL dialogues by constructing various machine learning-based models. Our results demonstrate that certain micro-level features strongly correlate with interactivity quality, like reference word (e.g., she, her, he), revealing new insights about the interaction between higher-level dialogue quality and lower-level linguistic signals. Our framework also provides a means to assess ESL communication, which is useful for language assessment.
翻译:我们提出了一个针对英语作为第二语言(ESL)使用者的互动对话评估框架。该框架收集对话层面的互动性标签(例如话题管理;共4个标签)以及微观层面的片段特征(例如反馈信号;共17个特征)。基于已标注数据,我们通过构建多种基于机器学习的模型,研究了微观层面特征如何影响ESL对话的(更高层面)互动性质量。我们的结果表明,某些微观特征(如指称词(例如she、her、he))与互动性质量存在强相关性,这揭示了高层对话质量与低层语言信号之间相互作用的新见解。本框架也为评估ESL交流能力提供了工具,对语言评估具有实用价值。