Existing approaches to algorithmic fairness aim to ensure equitable outcomes if human decision-makers comply perfectly with algorithmic decisions. However, perfect compliance with the algorithm is rarely a reality or even a desirable outcome in human-AI collaboration. Yet, recent studies have shown that selective compliance with fair algorithms can amplify discrimination relative to the prior human policy. As a consequence, ensuring equitable outcomes requires fundamentally different algorithmic design principles that ensure robustness to the decision-maker's (a priori unknown) compliance pattern. We define the notion of compliance-robustly fair algorithmic recommendations that are guaranteed to (weakly) improve fairness in decisions, regardless of the human's compliance pattern. We propose a simple optimization strategy to identify the best performance-improving compliance-robustly fair policy. However, we show that it may be infeasible to design algorithmic recommendations that are simultaneously fair in isolation, compliance-robustly fair, and more accurate than the human policy; thus, if our goal is to improve the equity and accuracy of human-AI collaboration, it may not be desirable to enforce traditional fairness constraints.

翻译：现有的算法公平性方法旨在确保当人类决策者完全遵循算法决策时能实现公平结果。然而在人机协作中，完全遵循算法既非现实也非理想目标。近期研究表明，对公平算法的选择性遵循可能相对于先前的人类政策放大歧视。因此，确保公平结果需要从根本上不同的算法设计原则——这些原则必须对决策者（先验未知的）遵循模式具有鲁棒性。我们定义了"遵循鲁棒公平"算法建议的概念，该建议能保证无论人类的遵循模式如何，都能（弱）改善决策公平性。我们提出了一种简单的优化策略来识别最优性能改进的遵循鲁棒公平政策。然而研究表明，设计同时满足孤立公平、遵循鲁棒公平且比人类政策更准确的算法建议可能不可行；因此，如果我们的目标是提升人机协作的公平性与准确性，强制传统公平约束可能并非可取之策。