Generate-then-rank is a widely used mechanism for text generation, where a generator produces multiple text candidates and a ranker chooses the best one among the text candidates. However, existing methods usually train the generator and the ranker individually, neglecting the mutual feedback that could further enhance the generation quality. To tackle this limitation, we propose JGR, a novel joint training algorithm that integrates the generator and the ranker in a single framework. JGR optimizes the generator with a hybrid objective that combines data likelihood and ranker reward, and trains the ranker with a contrastive loss that compares the generator outputs. By iteratively updating the generator and the ranker, JGR can effectively harmonize their learning and enhance their quality jointly. We evaluate JGR on various text generation tasks and demonstrate that it surpasses existing methods on four public datasets across three common generation scenarios. Our code and models are publicly available at https://github.com/microsoft/ProphetNet/tree/master/JGR.

翻译：“生成后排序”是文本生成中广泛采用的机制，即生成器产生多个候选文本，排序器从中选出最优结果。然而，现有方法通常单独训练生成器和排序器，忽略了能进一步提升生成质量的相互反馈。为解决这一局限，我们提出JGR，一种将生成器与排序器整合在统一框架中的新型联合训练算法。JGR通过结合数据似然和排序器奖励的混合目标优化生成器，并采用对比损失训练排序器以比较生成器输出。通过迭代更新生成器与排序器，JGR能有效协调两者的学习过程，并共同提升其质量。我们在多种文本生成任务上评估JGR，结果表明其在三个常见生成场景的四个公开数据集上均优于现有方法。我们的代码和模型已公开于https://github.com/microsoft/ProphetNet/tree/master/JGR。