Large language models (LLMs) have shown impressive capabilities in adapting to various tasks when provided with task-specific instructions. However, LLMs using standard decoding strategies often struggle with deviations from the inputs. Intuitively, compliant LLM outputs should reflect the information present in the input, which can be measured by point-wise mutual information (PMI) scores. Therefore, we propose Diver, a novel approach that enhances LLM Decoding through span-level PMI verification. During inference, Diver first identifies divergence steps that may lead to multiple candidate spans. Subsequently, it calculates the PMI scores by assessing the log-likelihood gains of the input if the candidate spans are generated. Finally, the optimal span is selected based on the PMI re-ranked output distributions. We evaluate our method across various downstream tasks, and empirical results demonstrate that Diver significantly outperforms existing decoding methods in both performance and versatility.

翻译：大语言模型（LLMs）在获得特定任务指令后，已展现出适应各类任务的卓越能力。然而，采用标准解码策略的LLM在处理输入偏差时常常面临困难。直观而言，符合要求的LLM输出应当反映输入中包含的信息，这一特性可通过逐点互信息（PMI）分数进行量化。为此，我们提出Diver——一种通过片段级PMI验证来增强LLM解码的新方法。在推理过程中，Diver首先识别可能导致多个候选片段的发散步骤；随后通过评估生成候选片段时输入的似然增益来计算PMI分数；最终根据PMI重排序的输出分布选择最优片段。我们在多种下游任务中对该方法进行评估，实证结果表明Diver在性能与泛化能力方面均显著优于现有解码方法。