Lexicon-based constrained decoding approaches aim to control the meaning or style of the generated text through certain target concepts. Existing approaches over-focus the targets themselves, leading to a lack of high-level reasoning about how to achieve them. However, human usually tackles tasks by following certain rules that not only focuses on the targets but also on semantically relevant concepts that induce the occurrence of targets. In this work, we present DECIDER, a rule-controllable decoding strategy for constrained language generation inspired by dual-system cognitive theory. Specifically, in DECIDER, a pre-trained language model (PLM) is equiped with a logic reasoner that takes high-level rules as input. Then, the DECIDER allows rule signals to flow into the PLM at each decoding step. Extensive experimental results demonstrate that DECIDER can effectively follow given rules to guide generation direction toward the targets in a more human-like manner.

翻译：摘要：基于词典的约束解码方法旨在通过特定目标概念控制生成文本的含义或风格。现有方法过度关注目标本身，缺乏关于如何实现目标的高级推理能力。然而，人类通常遵循特定规则处理任务，不仅关注目标本身，还会关注语义相关且能诱发目标实现的概念。在本工作中，我们提出DECIDER——一种受双系统认知理论启发的规则可控约束语言生成解码策略。具体而言，DECIDER为预训练语言模型配备了一个逻辑推理器，该推理器以高级规则作为输入。随后，DECIDER允许规则信号在每个解码步骤流入预训练语言模型。大量实验结果表明，DECIDER能够有效遵循给定规则，以更接近人类的方式引导生成方向朝向目标。