Large language models (LLMs) are now being considered and even deployed for applications that support high-stakes decision-making, such as recruitment and clinical decisions. While several methods have been proposed for measuring bias, there remains a gap between predictions, which are what the proposed methods consider, and how they are used to make decisions. In this work, we introduce Rank-Allocational-Based Bias Index (RABBI), a model-agnostic bias measure that assesses potential allocational harms arising from biases in LLM predictions. We compare RABBI and current bias metrics on two allocation decision tasks. We evaluate their predictive validity across ten LLMs and utility for model selection. Our results reveal that commonly-used bias metrics based on average performance gap and distribution distance fail to reliably capture group disparities in allocation outcomes, whereas RABBI exhibits a strong correlation with allocation disparities. Our work highlights the need to account for how models are used in contexts with limited resource constraints.

翻译：大型语言模型（LLMs）当前正被考虑甚至已部署于支持高风险决策的应用场景，如招聘和临床诊断。尽管已有多种测量偏见的方法被提出，但预测结果（即现有方法所考量的内容）与实际决策应用之间仍存在差距。本研究提出基于排序分配的偏见指数（RABBI），这是一种模型无关的偏见度量方法，用于评估LLM预测偏差可能引发的分配性危害。我们在两项分配决策任务中对比了RABBI与现有偏见指标，并通过对十个LLM的预测效度检验及模型选择效用评估发现：基于平均性能差距和分布距离的常用偏见指标无法可靠捕捉分配结果中的群体差异，而RABBI则与分配差异呈现强相关性。本研究强调在有限资源约束的决策场景中，必须考量模型的实际使用机制。