Diagnosing and mitigating changes in model fairness under distribution shift is an important component of the safe deployment of machine learning in healthcare settings. Importantly, the success of any mitigation strategy strongly depends on the structure of the shift. Despite this, there has been little discussion of how to empirically assess the structure of a distribution shift that one is encountering in practice. In this work, we adopt a causal framing to motivate conditional independence tests as a key tool for characterizing distribution shifts. Using our approach in two medical applications, we show that this knowledge can help diagnose failures of fairness transfer, including cases where real-world shifts are more complex than is often assumed in the literature. Based on these results, we discuss potential remedies at each step of the machine learning pipeline.

翻译：诊断和减轻分配转移中的公平性模式变化是安全部署保健环境中机器学习的一个重要部分。重要的是,任何缓解战略的成功都在很大程度上取决于转变的结构。尽管如此,对于如何实证评估实际中遇到的分配转移结构的讨论很少。在这项工作中,我们采用了一个因果框架,以激励有条件的独立测试作为分配转移特征的关键工具。我们采用两种医疗应用方法,表明这种知识可以帮助诊断公平转让的失败,包括现实世界的转变比文献中通常假设的更为复杂的情况。基于这些结果,我们讨论了机器学习管道每一步的潜在补救措施。