Conversational explainable artificial intelligence (ConvXAI) systems based on large language models (LLMs) have garnered significant interest from the research community in natural language processing (NLP) and human-computer interaction (HCI). Such systems can provide answers to user questions about explanations, have the potential to enhance users' comprehension and offer more information about the decision-making and generation processes of LLMs. Currently available ConvXAI systems are based on intent recognition rather than free chat. Thus, reliably grasping users' intentions in ConvXAI systems still presents a challenge, because there is a broad range of XAI methods to map requests onto and each of them can have multiple slots to take care of. In order to bridge this gap, we present CoXQL, the first dataset for user intent recognition in ConvXAI, covering 31 intents, seven of which require filling additional slots. Subsequently, we enhance an existing parsing approach by incorporating template validations, and conduct an evaluation of several LLMs on CoXQL using different parsing strategies. We conclude that the improved parsing approach (MP+) surpasses the performance of previous approaches. We also discover that intents with multiple slots remain highly challenging for LLMs.

翻译：基于大语言模型（LLM）的对话式可解释人工智能（ConvXAI）系统已引起自然语言处理（NLP）和人机交互（HCI）研究领域的广泛关注。此类系统能够回应用户关于解释的提问，有望提升用户的理解能力，并为LLM的决策与生成过程提供更丰富的信息。当前可用的ConvXAI系统多基于意图识别而非自由对话模式。由于需将用户请求映射至多种可解释人工智能方法，且每种方法可能涉及多个待填充的参数槽，在ConvXAI系统中可靠地捕捉用户意图仍面临挑战。为弥补这一空白，我们提出了CoXQL——首个面向ConvXAI用户意图识别的数据集，涵盖31种意图类型，其中七类需额外填充参数槽。基于此，我们通过引入模板验证机制改进现有解析方法，并采用不同解析策略在CoXQL上对多种LLM进行评估。实验表明，改进后的解析方法（MP+）性能优于现有方法。同时我们发现，涉及多参数槽的意图解析对LLM而言仍具较高挑战性。