Preference-based alignment objectives have been widely adopted, from RLHF-style pairwise learning in large language models to emerging applications in recommender systems. Yet, existing work rarely examines how Direct Preference Optimization (DPO) behaves under implicit feedback, where unobserved items are not reliable negatives. We conduct systematic experiments on multimodal sequential recommendation to compare common negative-selection strategies and their interaction with DPO training. Our central finding is that a simple modification, replacing deterministic hard negatives with stochastic sampling from a dynamic top-K candidate pool, consistently improves ranking performance. We attribute its effectiveness to two factors: (1) reducing erroneous suppressive gradients caused by false negatives, and (2) retaining informative hard signals while smoothing optimization via controlled stochasticity. With an optional sparse Mixture-of-Experts encoder for efficient capacity scaling, RoDPO achieves up to 5.25% NDCG@5 on three Amazon benchmarks, with nearly unchanged inference cost.

翻译：基于偏好的对齐目标已被广泛采用，从大型语言模型中基于强化学习人类反馈的成对学习，到推荐系统中的新兴应用。然而，现有研究很少探讨直接偏好优化（DPO）在隐式反馈下的表现——其中未观测到的项目并非可靠的负样本。我们针对多模态序列推荐开展了系统性实验，比较了常见的负样本选择策略及其与DPO训练的交互作用。核心发现是：一个简单的修改——将确定性硬负样本替换为从动态Top-K候选池中进行随机采样——能够持续提升排序性能。我们将有效性归因于两个因素：（1）减少由假负样本引起的错误抑制梯度；（2）在通过受控随机性平滑优化的同时保留信息量丰富的硬信号。通过可选的稀疏专家混合编码器实现高效容量扩展，RoDPO在三个亚马逊基准测试上取得了高达5.25%的NDCG@5提升，且推理成本几乎不变。