Large language models (LLMs) are increasingly used for recommendation reranking, but their listwise predictions can depend on the order in which candidates are presented. This creates a mismatch between the set-based nature of recommendation and the sequence-based computation of decoder-only LLMs, where permuting an otherwise identical candidate set can change item scores and final rankings. Such order sensitivity makes LLM-based rerankers difficult to rely on, since rankings may reflect prompt serialization rather than user preference. We propose InvariRank, a permutation-invariant listwise reranking framework that addresses this dependence at the architectural level. InvariRank blocks cross-candidate attention with a structured attention mask and negates position-induced scoring changes through shared positional framing under Rotary Positional Embeddings (RoPE). Combined with a listwise learning-to-rank objective, the model scores all candidates in a single forward pass, avoiding permutation-based invariance training objectives that require multiple permutations of a candidate set. Experiments on recommendation benchmarks show that InvariRank maintains competitive ranking effectiveness while producing stable rankings across candidate permutations. The results suggest that architectural invariance is a practical route to reliable and efficient LLM-based recommendation reranking. The source code is at https://github.com/ejbito/InvariRank.
翻译:大型语言模型(LLM)越来越多地被用于推荐重排任务,但其列表级预测依赖于候选项目的呈现顺序。这导致基于集合的推荐本质与仅解码器LLM的序列计算方式之间存在矛盾——对相同候选集进行排列会改变项目得分和最终排序结果。这种顺序敏感性使得基于LLM的重排器难以可靠使用,因为排序结果可能反映提示序列化模式而非用户偏好。本文提出InvariRank——一种排列不变的列表级重排框架,在架构层面解决顺序依赖问题。InvariRank通过结构化注意力掩码阻断跨候选注意力,并借助旋转位置编码(RoPE)下的共享位置框架消除位置导致的评分偏差。结合列表级学习排序目标,该模型通过单次前向传播完成所有候选项目的评分,避免了需要候选集多种排列的基于排列不变性的训练目标。推荐基准实验表明,InvariRank在保持竞争性排序效果的同时,能对候选排列产生稳定排序结果。研究结果表明,架构级不变性是实现可靠高效基于LLM推荐重排的实用途径。源代码发布在 https://github.com/ejbito/InvariRank。