This research paper explores the potential of Large Language Models (LLMs) to enhance speaking skills. We first present a novel LLM-based system, Comuniqa, for this task. We then take a humancentric approach to evaluate this system, comparing it with human experts. We also investigate the possibility of combining feedback from both LLM and human experts to enhance overall learning outcomes. We use purposive and random sampling for recruiting participants, categorizing them into three groups: those who use LLM-enabled apps for improving speaking skills, those guided by human experts for the same task and those who utilize both the LLM-enabled apps as well as the human experts. Using surveys, interviews, and actual study sessions, we provide a detailed perspective on the effectiveness of different learning modalities. Our preliminary findings suggest that while LLM-based systems have commendable accuracy, they lack human-level cognitive capabilities, both in terms of accuracy and empathy.

翻译：本研究论文探讨了大型语言模型（LLMs）在提升口语技能方面的潜力。我们首先提出了一种基于LLM的新型系统Comuniqa，用于完成这一任务。随后，我们采用以人为本的方法评估该系统，并与人类专家进行对比。我们还研究了结合LLM与人类专家反馈，以增强整体学习效果的可能性。我们采用目的性抽样和随机抽样招募参与者，并将其分为三组：使用支持LLM的应用提升口语技能的参与者、由人类专家指导完成相同任务的参与者，以及同时使用支持LLM的应用和人类专家的参与者。通过问卷调查、访谈和实际学习会话，我们对不同学习模式的有效性提供了详细视角。初步研究结果表明，尽管基于LLM的系统具有值得称赞的准确性，但它们在准确性和共情能力方面仍缺乏人类水平的认知能力。