Response generation is one of the critical components in task-oriented dialog systems. Existing studies have shown that large pre-trained language models can be adapted to this task. The typical paradigm of adapting such extremely large language models would be by fine-tuning on the downstream tasks which is not only time-consuming but also involves significant resources and access to fine-tuning data. Prompting \citep{schick2020exploiting} has been an alternative to fine-tuning in many NLP tasks. In our work, we explore the idea of using prompting for response generation in task-oriented dialog systems. Specifically, we propose an approach that performs \textit{contextual dynamic prompting} where the prompts are learnt from dialog contexts. We aim to distill useful prompting signals from the dialog context. On experiments with MultiWOZ 2.2 dataset \cite{zang2020multiwoz}, we show that contextual dynamic prompts improve response generation in terms of \textit{combined score} \cite{mehri-etal-2019-structured} by 3 absolute points, and a massive 20 points when dialog states are incorporated. Furthermore, human annotation on these conversations found that agents which incorporate context were preferred over agents with vanilla prefix-tuning.

翻译：回复生成是任务导向对话系统中的关键组成部分之一。现有研究表明，大规模预训练语言模型可以适用于该任务。适应此类超大规模语言模型的典型范式是通过在下游任务上进行微调，这不仅耗时，还需要大量资源和对微调数据的访问。提示（prompting）已成为许多自然语言处理任务中微调的替代方案。在本研究中，我们探索了在任务导向对话系统中使用提示进行回复生成的想法。具体而言，我们提出了一种执行“情境动态提示”的方法，其中提示从对话情境中学习得到。我们旨在从对话情境中提取有用的提示信号。在MultiWOZ 2.2数据集上的实验中，我们发现情境动态提示在综合得分上将回复生成性能提升了3个绝对百分点，并且当对话状态被纳入时，提升幅度高达20个百分点。此外，对这些对话进行的人工标注发现，融入情境的代理优于使用原始前缀调优的代理。