A growing literature on human-AI decision-making investigates strategies for combining human judgment with statistical models to improve decision-making. Research in this area often evaluates proposed improvements to models, interfaces, or workflows by demonstrating improved predictive performance on "ground truth" labels. However, this practice overlooks a key difference between human judgments and model predictions. Whereas humans reason about broader phenomena of interest in a decision -- including latent constructs that are not directly observable, such as disease status, the "toxicity" of online comments, or future "job performance" -- predictive models target proxy labels that are readily available in existing datasets. Predictive models' reliance on simplistic proxies makes them vulnerable to various sources of statistical bias. In this paper, we identify five sources of target variable bias that can impact the validity of proxy labels in human-AI decision-making tasks. We develop a causal framework to disentangle the relationship between each bias and clarify which are of concern in specific human-AI decision-making tasks. We demonstrate how our framework can be used to articulate implicit assumptions made in prior modeling work, and we recommend evaluation strategies for verifying whether these assumptions hold in practice. We then leverage our framework to re-examine the designs of prior human subjects experiments that investigate human-AI decision-making, finding that only a small fraction of studies examine factors related to target variable bias. We conclude by discussing opportunities to better address target variable bias in future research.

翻译：越来越多关于人机协同决策的研究致力于探索将人工判断与统计模型相结合的策略以提升决策质量。该领域的研究通常通过展示模型在"真实标注"上预测性能的提升，来评估对模型、交互界面或工作流程的改进方案。然而，这种做法忽视了人工判断与模型预测之间的关键差异：人类会权衡决策中更广泛的现象维度——包括无法直接观测的潜变量，如疾病状态、网络评论的"毒性"或未来"工作绩效"——而预测模型则仅针对现有数据集中易于获取的代理标签进行建模。预测模型对简化代理标签的依赖使其易受多种统计偏差影响。本文识别出影响人机协同决策任务中代理标签有效性的五类目标变量偏差来源，并构建因果框架厘清各类偏差间的关联机制，阐明特定任务场景中需重点关注的偏差类型。我们演示了该框架如何揭示既往建模研究中隐含的假设条件，并提出评估策略以验证这些假设在实践中的适用性。通过重新审视前人开展的人机协同决策行为实验设计，本研究发现仅有少数研究涉及与目标变量偏差相关的因素。最后，我们探讨了未来研究中更有效应对目标变量偏差的改进方向。