When machine-learning algorithms are used in high-stakes decisions, we want to ensure that their deployment leads to fair and equitable outcomes. This concern has motivated a fast-growing literature that focuses on diagnosing and addressing disparities in machine predictions. However, many machine predictions are deployed to assist in decisions where a human decision-maker retains the ultimate decision authority. In this article, we therefore consider in a formal model and in a lab experiment how properties of machine predictions affect the resulting human decisions. In our formal model of statistical decision-making, we show that the inclusion of a biased human decision-maker can revert common relationships between the structure of the algorithm and the qualities of resulting decisions. Specifically, we document that excluding information about protected groups from the prediction may fail to reduce, and may even increase, ultimate disparities. In the lab experiment, we demonstrate how predictions informed by gender-specific information can reduce average gender disparities in decisions. While our concrete theoretical results rely on specific assumptions about the data, algorithm, and decision-maker, and the experiment focuses on a particular prediction task, our findings show more broadly that any study of critical properties of complex decision systems, such as the fairness of machine-assisted human decisions, should go beyond focusing on the underlying algorithmic predictions in isolation.

翻译：当机器学习算法用于高风险决策时，我们需确保其部署能导向公平公正的结果。这一关切催生了大量聚焦于诊断与消除机器学习预测差异性的研究文献。然而，许多机器学习预测被部署用于辅助决策，此时人类决策者仍保留最终决策权。因此，本文通过形式化模型与实验室实验，研究机器学习预测的特性如何影响由此产生的人类决策。在统计决策的形式化模型中，我们证明：引入存在偏见的人类决策者可能逆转算法结构与最终决策质量之间的常见关系。具体而言，我们观察到，从预测中剔除受保护群体的信息不仅无法减少最终差异，甚至可能加剧这种差异。在实验室实验中，我们展示了基于性别特征信息的预测如何降低决策中平均性别差异。尽管本文的具体理论结果依赖于对数据、算法及决策者的特定假设，且实验聚焦于特定预测任务，但我们的研究更广泛地表明：对复杂决策系统关键属性（如机器学习辅助人类决策的公平性）的研究，不应仅局限于对底层算法预测的孤立分析。