Direct Preference Optimization (DPO) is a powerful paradigm for aligning Large Language Models (LLMs) to human preferences in Machine Translation (MT), but current methods are hindered by two fundamental challenges: (1) flawed reward signals from Quality Estimation (QE) models that overlook critical errors like translation hallucination, and (2) inefficient data utilization that discards valuable learning signals by selecting only a single win-loss pair. To address these limitations, we introduce M^2PO: Multi-Pair, Multi-Perspective Preference Optimization. Our framework integrates a multi-perspective reward engine that creates a more robust signal by combining two key viewpoints: a new hallucination penalty for factuality, and an innovative dynamic quality score that adaptively fuses external evaluations with the model's own evolving judgment. This is synergistically paired with a multi-pair construction strategy that systematically creates a comprehensive set of preference pairs from the entire pool of translation candidates. This synergistic approach ensures the model learns from a richer spectrum of quality trade-offs, leading to more robust and faithful translations. On challenging WMT21-22 benchmarks, M^2PO substantially outperforms existing preference optimization methods and demonstrates highly competitive performance against leading proprietary LLMs.

翻译：直接偏好优化（DPO）是将大语言模型（LLM）与人类偏好对齐以用于机器翻译（MT）的强大范式，但现有方法受限于两个基本挑战：（1）来自质量评估（QE）模型的奖励信号存在缺陷，其忽略了如翻译幻觉等关键错误；（2）数据利用效率低下，仅选取单一胜负配对而丢弃了有价值的学习信号。为应对这些局限，我们提出了M²PO：多配对、多视角偏好优化框架。该框架集成了一个多视角奖励引擎，通过融合两个关键视角生成更稳健的奖励信号：其一是针对事实性的新型幻觉惩罚机制，其二是创新的动态质量评分，该评分自适应地融合外部评估与模型自身演化的判断。该引擎与多配对构建策略协同工作，系统地从全部翻译候选中构建出全面的偏好配对集合。这种协同方法确保模型能从更丰富的质量权衡谱中学习，从而产生更稳健、更忠实的翻译。在具有挑战性的WMT21-22基准测试中，M²PO显著优于现有偏好优化方法，并在与领先的专有大语言模型对比中展现出高度竞争力。