A critical component of a successful language generation pipeline is the decoding algorithm. However, the general principles that should guide the choice of a decoding algorithm remain unclear. Previous works only compare decoding algorithms in narrow scenarios, and their findings do not generalize across tasks. We argue that the misalignment between the model's likelihood and the task-specific notion of utility is the key factor to understanding the effectiveness of decoding algorithms. To structure the discussion, we introduce a taxonomy of misalignment mitigation strategies (MMSs), providing a unifying view of decoding as a tool for alignment. The MMS taxonomy groups decoding algorithms based on their implicit assumptions about likelihood--utility misalignment, yielding general statements about their applicability across tasks. Specifically, by analyzing the correlation between the likelihood and the utility of predictions across a diverse set of tasks, we provide empirical evidence supporting the proposed taxonomy and a set of principles to structure reasoning when choosing a decoding algorithm. Crucially, our analysis is the first to relate likelihood-based decoding algorithms with algorithms that rely on external information, such as value-guided methods and prompting, and covers the most diverse set of tasks to date. Code, data, and models are available at https://github.com/epfl-dlab/understanding-decoding.

翻译：成功的语言生成流程中，一个关键组成部分是解码算法。然而，指导解码算法选择的通用原则仍不明确。以往研究仅在狭窄场景下比较解码算法，其结论难以跨任务泛化。我们认为，模型似然与任务特定效用概念之间的错位，是理解解码算法效果的关键因素。为结构化讨论，我们引入错位缓解策略（MMS）分类体系，将解码统一视为对齐工具。MMS分类体系根据解码算法对似然-效用错位的隐含假设进行分组，从而得出关于其跨任务适用性的通用结论。具体而言，通过分析多样任务中预测的似然与效用之间相关性，我们提供了支持该分类体系的实证证据，并提出一套用于结构化选择解码算法推理过程的原则。关键的是，我们的分析首次将基于似然的解码算法与依赖外部信息的算法（如价值引导方法和提示）相关联，并覆盖了迄今最广泛的任务集合。代码、数据和模型详见https://github.com/epfl-dlab/understanding-decoding。