Text generation with beam search has proven successful in a wide range of applications. We point out that, though largely overlooked in the literature, the commonly-used implementation of beam decoding (e.g., Hugging Face Transformers and fairseq) uses a first come, first served heuristic: it keeps a set of already completed sequences over time steps and stops when the size of this set reaches the beam size. Based on this finding, we introduce a patience factor, a simple modification to this beam decoding implementation, that generalizes the stopping criterion and provides flexibility to the depth of search. Empirical results demonstrate that adjusting this patience factor improves decoding performance of strong pretrained models on news text summarization and machine translation over diverse language pairs, with a negligible inference slowdown. Our approach only modifies one line of code and can be thus readily incorporated in any implementation. Further, we find that different versions of beam decoding result in large performance differences in summarization, demonstrating the need for clarity in specifying the beam search implementation in research work. Our code will be available upon publication.

翻译：波束搜索在文本生成领域已成功应用于众多场景。我们指出，尽管文献中常被忽视，但普遍使用的波束解码实现（例如Hugging Face Transformers和fairseq）采用了一种“先到先得”的启发式策略：它在时间步上持续维护一组已完成的序列，当这组序列的数量达到波束宽度时便停止搜索。基于此发现，我们引入了一个“耐心因子”（patience factor），作为对该波束解码实现的简单修改，它推广了终止准则，并为搜索深度提供了灵活性。实验结果表明，在新闻文本摘要和跨语言机器翻译任务中，调整这一耐心因子能够显著提升强预训练模型的解码性能，而推理速度的下降微乎其微。我们的方法仅需修改一行代码，因此可轻松集成到任何实现中。此外，我们发现不同版本的波束解码在摘要任务上会导致巨大的性能差异，这凸显了在研究工作中明确说明波束搜索实现的必要性。我们的代码将在论文发表后公开。