Robot systems in education can leverage Large language models' (LLMs) natural language understanding capabilities to provide assistance and facilitate learning. This paper proposes a multimodal interactive robot (PhysicsAssistant) built on YOLOv8 object detection, cameras, speech recognition, and chatbot using LLM to provide assistance to students' physics labs. We conduct a user study on ten 8th-grade students to empirically evaluate the performance of PhysicsAssistant with a human expert. The Expert rates the assistants' responses to student queries on a 0-4 scale based on Bloom's taxonomy to provide educational support. We have compared the performance of PhysicsAssistant (YOLOv8+GPT-3.5-turbo) with GPT-4 and found that the human expert rating of both systems for factual understanding is the same. However, the rating of GPT-4 for conceptual and procedural knowledge (3 and 3.2 vs 2.2 and 2.6, respectively) is significantly higher than PhysicsAssistant (p < 0.05). However, the response time of GPT-4 is significantly higher than PhysicsAssistant (3.54 vs 1.64 sec, p < 0.05). Hence, despite the relatively lower response quality of PhysicsAssistant than GPT-4, it has shown potential for being used as a real-time lab assistant to provide timely responses and can offload teachers' labor to assist with repetitive tasks. To the best of our knowledge, this is the first attempt to build such an interactive multimodal robotic assistant for K-12 science (physics) education.

翻译：教育领域的机器人系统可利用大语言模型（LLMs）的自然语言理解能力，为学生提供辅助并促进学习。本文提出了一种基于YOLOv8目标检测、摄像头、语音识别及大语言模型聊天机器人构建的多模态交互式机器人（PhysicsAssistant），用于辅助学生进行物理实验。我们通过对十名八年级学生开展用户研究，将PhysicsAssistant与人类专家的表现进行实证评估。专家根据布鲁姆教育目标分类法，以0-4分制对助手回应学生提问的质量进行评分，以衡量其教育支持效果。我们比较了PhysicsAssistant（YOLOv8+GPT-3.5-turbo）与GPT-4的表现，发现两者在事实性知识理解方面获得的人类专家评分相同。然而，GPT-4在概念性知识与程序性知识方面的评分（分别为3分与3.2分）显著高于PhysicsAssistant（2.2分与2.6分）（p < 0.05）。但GPT-4的响应时间（3.54秒）显著高于PhysicsAssistant（1.64秒）（p < 0.05）。因此，尽管PhysicsAssistant的响应质量相对低于GPT-4，但其在作为实时实验助手提供及时响应、减轻教师处理重复性任务的负担方面展现出潜力。据我们所知，这是为K-12阶段科学（物理）教育构建此类交互式多模态机器人助手的首次尝试。