When applied to question answering and other text generation tasks, language models (LMs) may be queried generatively (by sampling answers from their output distribution) or discriminatively (by using them to score or rank a set of candidate outputs). These procedures sometimes yield very different predictions. How do we reconcile mutually incompatible scoring procedures to obtain coherent LM predictions? We introduce a new, a training-free, game-theoretic procedure for language model decoding. Our approach casts language model decoding as a regularized imperfect-information sequential signaling game - which we term the CONSENSUS GAME - in which a GENERATOR seeks to communicate an abstract correctness parameter using natural language sentences to a DISCRIMINATOR. We develop computational procedures for finding approximate equilibria of this game, resulting in a decoding algorithm we call EQUILIBRIUM-RANKING. Applied to a large number of tasks (including reading comprehension, commonsense reasoning, mathematical problem-solving, and dialog), EQUILIBRIUM-RANKING consistently, and sometimes substantially, improves performance over existing LM decoding procedures - on multiple benchmarks, we observe that applying EQUILIBRIUM-RANKING to LLaMA-7B outperforms the much larger LLaMA-65B and PaLM-540B models. These results highlight the promise of game-theoretic tools for addressing fundamental challenges of truthfulness and consistency in LMs.

翻译：在问答及其他文本生成任务中，语言模型（LM）可通过生成式方法（从其输出分布中采样答案）或判别式方法（用于对候选输出进行评分或排序）进行查询。这些过程有时会产生截然不同的预测。如何调和相互不兼容的评分过程以获得连贯的LM预测？我们提出了一种新的、无需训练的、基于博弈论的语言模型解码方法。该方法将语言模型解码形式化为一个正则化的不完全信息序贯信号博弈——我们称之为“共识博弈”——其中生成器试图通过自然语言句子向判别器传递一个抽象的语义正确性参数。我们开发了求解该博弈近似均衡的计算程序，进而提出了一种解码算法：均衡排序。应用于大量任务（包括阅读理解、常识推理、数学问题求解和对话）时，均衡排序始终如一地，有时甚至显著地优于现有LM解码过程——在多个基准测试中，我们将均衡排序应用于LLaMA-7B后，其性能超越了规模更大的LLaMA-65B和PaLM-540B模型。这些结果凸显了博弈论工具在解决语言模型真实性与一致性等基本挑战方面的潜力。