Automated decision-making (ADM) systems are being deployed across a diverse range of critical problem areas such as social welfare and healthcare. Recent work highlights the importance of causal ML models in ADM systems, but implementing them in complex social environments poses significant challenges. Research on how these challenges impact the performance in specific downstream decision-making tasks is limited. Addressing this gap, we make use of a comprehensive real-world dataset of jobseekers to illustrate how the performance of a single CATE model can vary significantly across different decision-making scenarios and highlight the differential influence of challenges such as distribution shifts on predictions and allocations.

翻译：自动决策系统正被广泛应用于社会福利和医疗保健等关键领域。近期研究强调了因果机器学习模型在自动决策系统中的重要性，但在复杂社会环境中部署这些模型仍面临重大挑战。关于这些挑战如何影响特定下游决策任务性能的研究尚不充分。为填补这一空白，我们利用一个全面的求职者真实数据集，展示了单一CATE模型在不同决策场景下的性能如何显著变化，并揭示了分布偏移等挑战对预测和分配结果的差异化影响。