Standard decoding approaches for conditional text generation tasks typically search for an output hypothesis with high model probability, but this may not yield the best hypothesis according to human judgments of quality. Reranking to optimize for "downstream" metrics can better optimize for quality, but many metrics of interest are computed with pre-trained language models, which are slow to apply to large numbers of hypotheses. We explore an approach for reranking hypotheses by using Transformers to efficiently encode lattices of generated outputs, a method we call EEL. With a single Transformer pass over the entire lattice, we can approximately compute a contextualized representation of each token as if it were only part of a single hypothesis in isolation. We combine this approach with a new class of token-factored rerankers (TFRs) that allow for efficient extraction of high reranker-scoring hypotheses from the lattice. Empirically, our approach incurs minimal degradation error compared to the exponentially slower approach of encoding each hypothesis individually. When applying EEL with TFRs across three text generation tasks, our results show both substantial speedup compared to naive reranking and often better performance on downstream metrics than comparable approaches.

翻译：面向条件文本生成任务的标准解码方法通常搜索具有高模型概率的输出假设，但这可能无法根据人类质量评判产生最优假设。为优化"下游"指标而进行重排序能更好地提升质量，但许多关键指标需通过预训练语言模型计算，这会显著降低对大量假设的处理速度。我们探索了一种利用Transformer高效编码生成输出格结构的重排序方法，称为EEL。通过对整个格结构执行单次Transformer前向传播，我们可近似计算出每个标记的上下文表示，仿佛该标记仅属于独立处理的单一假设。我们将该方案与新型令牌因子化重排序器（TFRs）相结合，实现从格结构中高效提取高重排序得分的假设。实验表明，相比对每个假设独立编码的指数级慢速方法，我们的方法仅引入极小的退化误差。将EEL与TFRs应用于三项文本生成任务时，实验结果显示，相较于朴素重排序方法，该方法不仅显著提升处理速度，在下游指标上也往往优于同类方法。