Zero-shot dialogue understanding aims to enable dialogue to track the user's needs without any training data, which has gained increasing attention. In this work, we investigate the understanding ability of ChatGPT for zero-shot dialogue understanding tasks including spoken language understanding (SLU) and dialogue state tracking (DST). Experimental results on four popular benchmarks reveal the great potential of ChatGPT for zero-shot dialogue understanding. In addition, extensive analysis shows that ChatGPT benefits from the multi-turn interactive prompt in the DST task but struggles to perform slot filling for SLU. Finally, we summarize several unexpected behaviors of ChatGPT in dialogue understanding tasks, hoping to provide some insights for future research on building zero-shot dialogue understanding systems with Large Language Models (LLMs).
翻译:零样本对话理解旨在使对话系统无需任何训练数据即可追踪用户需求,这一方向日益受到关注。本研究探讨了ChatGPT在零样本对话理解任务(包括口语语言理解(SLU)和对话状态跟踪(DST))中的理解能力。在四个主流基准测试上的实验结果表明,ChatGPT在零样本对话理解方面展现出巨大潜力。此外,广泛的分析显示,ChatGPT在DST任务中受益于多轮交互式提示,但在SLU的任务中难以执行槽位填充。最后,我们总结了ChatGPT在对话理解任务中若干意外行为,以期为未来基于大语言模型(LLMs)构建零样本对话理解系统的研究提供启示。