Constrained decoding approaches aim to control the meaning or style of text generated by a Pre-trained Language Model (PLM) using specific target words during inference. However, these methods often guide plausible continuations by greedily selecting targets, which, while completing the task, may disrupt the natural patterns of human language generation. In this work, we propose a novel decoding framework, DECIDER, which enables us to program rules on how we complete tasks to control a PLM. Differing from previous work, our framework transforms the encouragement of target words into the encouragement of all words that satisfy the rule. Specifically, DECIDER is a dual system where a PLM is equipped with a First-OrderLogic (FOL) reasoner to express and evaluate the rules, and a decision function to merge the outputs from both systems to steer the generation. Experiments on CommonGen and PersonaChat demonstrate that DECIDER can effectively follow given rules to achieve generation tasks in a more human-like manner.

翻译：约束解码方法旨在通过推理过程中使用特定目标词来控制预训练语言模型（PLM）生成的文本含义或风格。然而，这些方法通常通过贪婪地选择目标词来引导合理的后续生成，这虽然能完成任务，却可能破坏人类语言生成的自然模式。本文提出了一种新颖的解码框架DECIDER，使我们能够对如何完成任务进行规则编程，从而控制PLM。与先前工作不同，我们的框架将鼓励目标词转化为鼓励所有满足规则的词。具体而言，DECIDER是一个双系统框架，其中PLM配备了一个一阶逻辑（FOL）推理器来表达和评估规则，以及一个决策函数来融合两个系统的输出以引导生成。在CommonGen和PersonaChat数据集上的实验表明，DECIDER能够有效遵循给定规则，以更接近人类的方式完成生成任务。