Decision-making with information displays is a key focus of research in areas like explainable AI, human-AI teaming, and data visualization. However, what constitutes a decision problem, and what is required for an experiment to be capable of concluding that human decisions are flawed in some way, remain open to speculation. We present a widely applicable definition of a decision problem synthesized from statistical decision theory and information economics. We argue that to attribute loss in human performance to forms of bias, an experiment must provide participants with the information that a rational agent would need to identify the normative decision. We evaluate the extent to which recent evaluations of decision-making from the literature on AI-assisted decisions achieve this criteria. We find that only 10 (26\%) of 39 studies that claim to identify biased behavior present participants with sufficient information to characterize their behavior as deviating from good decision-making in at least one treatment condition. We motivate the value of studying well-defined decision problems by describing a characterization of performance losses they allow us to conceive. In contrast, the ambiguities of a poorly communicated decision problem preclude normative interpretation. We conclude with recommendations for practice.

翻译：信息显示下的决策制定是可解释人工智能、人机协作及数据可视化等领域的研究重点。然而，何谓决策问题，以及实验需要满足何种条件才能得出人类决策存在缺陷的结论，至今仍无定论。本文从统计决策理论与信息经济学中提炼出一个具有广泛适用性的决策问题定义。我们认为，要将人类决策绩效损失归因于各类偏差，实验必须为参与者提供理性主体识别规范性决策所需的信息。我们评估了近期人工智能辅助决策文献中若干决策评估研究满足该标准的程度。研究发现，在39项宣称识别出偏差行为的研究中，仅有10项（26%）在至少一个处理条件下为参与者提供了足够信息，足以将他们的行为定性为偏离优质决策。通过描述正确定义决策问题所能揭示的绩效损失特征，我们论证了研究这类问题的价值。反之，表述不清的决策问题存在的歧义将妨碍规范性解读。最后，我们提出了实践建议。