Designing natural language interfaces has historically required collecting supervised data to translate user requests into carefully designed intent representations. This requires enumerating and labeling a long tail of user requests, which is challenging. At the same time, large language models (LLMs) encode knowledge about goals and plans that can help conversational assistants interpret user requests requiring numerous steps to complete. We introduce an approach to handle complex-intent-bearing utterances from a user via a process of hierarchical natural language decomposition and interpretation. Our approach uses a pre-trained language model to decompose a complex utterance into a sequence of simpler natural language steps and interprets each step using the language-to-program model designed for the interface. To test our approach, we collect and release DeCU -- a new NL-to-program benchmark to evaluate Decomposition of Complex Utterances. Experiments show that the proposed approach enables the interpretation of complex utterances with almost no complex training data, while outperforming standard few-shot prompting approaches.
翻译:设计自然语言接口历来需要收集监督数据,将用户请求转换为精心设计的意图表示。这需要枚举并标注用户请求的"长尾"分布,具有较大挑战性。与此同时,大型语言模型(LLMs)编码了关于目标与计划的知识,能够帮助对话助手解释需要多步骤完成的用户请求。我们提出一种方法,通过层级化的自然语言分解与解释流程,处理用户带有复杂意图的话语。该方法采用预训练语言模型将复杂话语分解为一系列更简单的自然语言步骤,并使用为接口设计的语言-程序模型解释每个步骤。为测试该方法,我们收集并发布了DeCU——一个用于评估复杂话语分解的新型自然语言-程序基准数据集。实验表明,所提方法几乎无需复杂训练数据即可解释复杂话语,性能优于标准小样本提示方法。