If you tell a learning model that you prefer an alternative $a$ over another alternative $b$, then you probably expect the model to be monotone, that is, the valuation of $a$ increases, and that of $b$ decreases. Yet, perhaps surprisingly, many widely deployed comparison-based preference learning models, including large language models, fail to have this guarantee. Until now, the only comparison-based preference learning algorithms that were proved to be monotone are the Generalized Bradley-Terry models. Yet, these models are unable to generalize to uncompared data. In this paper, we advance the understanding of the set of models with generalization ability that are monotone. Namely, we propose a new class of Linear Generalized Bradley-Terry models with Diffusion Priors, and identify sufficient conditions on alternatives' embeddings that guarantee monotonicity. Our experiments show that this monotonicity is far from being a general guarantee, and that our new class of generalizing models improves accuracy, especially when the dataset is limited.

翻译：如果你告诉一个学习模型你更偏好备选方案$a$而非备选方案$b$，那么你很可能期望该模型具有单调性，即对$a$的评估值上升，而对$b$的评估值下降。然而或许令人惊讶的是，许多广泛应用的比较型偏好学习模型（包括大语言模型）都无法保证这一性质。迄今为止，唯一被证明具有单调性的比较型偏好学习算法是广义布拉德利-特里模型。但这些模型无法泛化到未比较的数据。本文推进了对具有泛化能力且保持单调性的模型集合的理解。具体而言，我们提出了一类新的带扩散先验的线性广义布拉德利-特里模型，并确定了保证单调性的备选方案嵌入充分条件。实验表明，这种单调性远非普遍保证，而我们提出的新型泛化模型能提升预测准确度，在数据集有限时尤为明显。