In large language model (LLM)-based recommendation systems, direct preference optimization (DPO) effectively aligns recommendations with user preferences, requiring multi-negative objective functions to leverage abundant implicit-feedback negatives and sharpen preference boundaries. However, our empirical analyses reveal a counterintuitive phenomenon, preference optimization collapse, where increasing the number of negative samples can lead to performance degradation despite a continuously decreasing training loss. We further theoretically demonstrate that this collapse arises from gradient suppression, caused by the dominance of easily discriminable negatives over boundary-critical negatives that truly define user preference boundaries. As a result, boundary-relevant signals are under-optimized, weakening the model's decision boundary. Motivated by these observations, we propose DynamicPO (Dynamic Preference Optimization), a lightweight and plug-and-play framework comprising two adaptive mechanisms: Dynamic Boundary Negative Selection, which identifies and prioritizes informative negatives near the model's decision boundary, and Dual-Margin Dynamic beta Adjustment, which calibrates optimization strength per sample according to boundary ambiguity. Extensive experiments on three public datasets show that DynamicPO effectively prevents optimization collapse and improves recommendation accuracy on multi-negative preference optimization methods, with negligible computational overhead. Our code and datasets are available at https://github.com/xingyuHuxingyu/DynamicPO.

翻译：在大语言模型（LLM）驱动的推荐系统中，直接偏好优化（DPO）方法能够有效对齐推荐结果与用户偏好，通过引入多负样本目标函数来充分利用丰富的隐式负反馈信号并锐化偏好边界。然而，我们的实证分析揭示了反直觉的"偏好优化崩溃"现象：尽管训练损失持续降低，增加负样本数量反而会导致性能下降。我们进一步从理论上证明，这种崩溃源于梯度抑制机制——易于区分的负样本主导了梯度更新，抑制了真正定义用户偏好边界的边界关键负样本的优化。因此，与边界相关的信号得不到充分优化，削弱了模型的决策边界。基于这些发现，我们提出DynamicPO（动态偏好优化）——一种轻量级即插即用框架，包含两种自适应机制：动态边界负样本选择（可识别并优先处理模型决策边界附近的富含信息负样本），以及双边际动态β调整（根据边界模糊度校准每个样本的优化强度）。在三个公开数据集上的大量实验表明，DynamicPO能够有效防止优化崩溃，并在多负样本偏好优化方法基础上提升推荐准确率，且计算开销可忽略不计。我们的代码和数据集已在https://github.com/xingyuHuxingyu/DynamicPO 公开。