Decision-making with information displays is a key focus of research in areas like explainable AI, human-AI teaming, and data visualization. However, what constitutes a decision problem, and what is required for an experiment to be capable of concluding that human decisions are flawed in some way, remain open to speculation. We present a widely applicable definition of a decision problem synthesized from statistical decision theory and information economics. We argue that to attribute loss in human performance to forms of bias, an experiment must provide participants with the information that a rational agent would need to identify the normative decision. We evaluate the extent to which recent evaluations of decision-making from the literature on AI-assisted decisions achieve this criteria. We find that only 6 (17\%) of 35 studies that claim to identify biased behavior present participants with sufficient information to characterize their behavior as deviating from good decision-making. We motivate the value of studying well-defined decision problems by describing a characterization of performance losses they allow us to conceive. In contrast, the ambiguities of a poorly communicated decision problem preclude normative interpretation. We conclude with recommendations for practice.

翻译：信息展示下的决策制定是可解释人工智能、人机协作及数据可视化等领域的核心研究焦点。然而，究竟何为决策问题，以及一项实验需满足何种条件才能得出人类决策存在缺陷的结论，目前仍属未解之谜。本文整合统计决策理论与信息经济学，提出一种具有广泛适用性的决策问题定义。我们认为，要将人类决策表现损失归因于某种偏差形式，实验必须提供理性代理人识别规范性决策所需的信息。我们评估了近期人工智能辅助决策文献中关于决策制定的研究在多大程度上满足这一标准。结果发现，在35篇声称识别出偏差行为的研究中，仅有6篇（17%）向参与者提供了充分信息，足以将其行为特征描述为偏离良好决策。通过描述明确定义的决策问题所允许我们构想的性能损失特征，我们论证了研究此类问题的价值。相反，表达含糊的决策问题中的歧义性会阻碍规范性解释。最后，我们提出实践建议。