Lexicon-based constrained decoding approaches aim to control the meaning or style of the generated text through certain target concepts. Existing approaches over-focus the targets themselves, leading to a lack of high-level reasoning about how to achieve them. However, human usually tackles tasks by following certain rules that not only focuses on the targets but also on semantically relevant concepts that induce the occurrence of targets. In this work, we present DECIDER, a rule-controllable decoding strategy for constrained language generation inspired by dual-system cognitive theory. Specifically, in DECIDER, a pre-trained language model (PLM) is equiped with a logic reasoner that takes high-level rules as input. Then, the DECIDER allows rule signals to flow into the PLM at each decoding step. Extensive experimental results demonstrate that DECIDER can effectively follow given rules to guide generation direction toward the targets in a more human-like manner.

翻译：基于词典的受限解码方法旨在通过特定目标概念控制生成文本的语义或风格。现有方法过度关注目标本身，缺乏对如何实现目标的高层推理。然而，人类通常通过遵循某些规则来完成任务，这些规则不仅关注目标本身，还关注语义相关且能诱发目标出现的概念。本文提出DECIDER，一种受双系统认知理论启发的规则可控语言生成解码策略。具体而言，DECIDER为预训练语言模型（PLM）配备了一个以高层规则作为输入的逻辑推理器，使规则信号能在每个解码步骤中流入PLM。大量实验结果表明，DECIDER能够有效遵循给定规则，以更接近人类的方式引导生成方向向目标靠拢。