Text-based dialogues are now widely used to solve real-world problems. In cases where solution strategies are already known, they can sometimes be codified into workflows and used to guide humans or artificial agents through the task of helping clients. We introduce a new problem formulation that we call Workflow Discovery (WD) in which we are interested in the situation where a formal workflow may not yet exist. Still, we wish to discover the set of actions that have been taken to resolve a particular problem. We also examine a sequence-to-sequence (Seq2Seq) approach for this novel task. We present experiments where we extract workflows from dialogues in the Action-Based Conversations Dataset (ABCD). Since the ABCD dialogues follow known workflows to guide agents, we can evaluate our ability to extract such workflows using ground truth sequences of actions. We propose and evaluate an approach that conditions models on the set of possible actions, and we show that using this strategy, we can improve WD performance. Our conditioning approach also improves zero-shot and few-shot WD performance when transferring learned models to unseen domains within and across datasets. Further, on ABCD a modified variant of our Seq2Seq method achieves state-of-the-art performance on related but different problems of Action State Tracking (AST) and Cascading Dialogue Success (CDS) across many evaluation metrics.
翻译:文本型对话现已广泛用于解决现实问题。当解决方案策略已知时,这些策略有时可被编码为工作流,用于指导人类或人工智能体帮助客户完成任务。我们提出一种称为工作流发现(WD)的新问题形式,该问题关注尚未存在正式工作流的情景,但希望发现解决特定问题所采取的一系列动作。我们还针对这一新颖任务探索了序列到序列(Seq2Seq)方法。我们展示了从基于动作的对话数据集(ABCD)中提取工作流的实验。由于ABCD对话遵循已知的工作流来指导智能体,我们能够通过真实动作序列评估提取此类工作流的能力。我们提出并评估了一种将模型条件化于可能动作集合的方法,并证明使用该策略可提升工作流发现性能。我们的条件化方法在将已训练模型迁移至数据集内外未见领域时,还能改进零样本和少样本场景下的工作流发现表现。此外,在ABCD数据集上,我们Seq2Seq方法的改进变体在动作状态追踪(AST)和级联对话成功(CDS)相关但不同的问题上,于多项评估指标中均达到了最先进性能。