We are interested in developing a data-driven method to evaluate race-induced biases in law enforcement systems. While the recent works have addressed this question in the context of police-civilian interactions using police stop data, they have two key limitations. First, bias can only be properly quantified if true criminality is accounted for in addition to race, but it is absent in prior works. Second, law enforcement systems are multi-stage and hence it is important to isolate the true source of bias within the "causal chain of interactions" rather than simply focusing on the end outcome; this can help guide reforms. In this work, we address these challenges by presenting a multi-stage causal framework incorporating criminality. We provide a theoretical characterization and an associated data-driven method to evaluate (a) the presence of any form of racial bias, and (b) if so, the primary source of such a bias in terms of race and criminality. Our framework identifies three canonical scenarios with distinct characteristics: in settings like (1) airport security, the primary source of observed bias against a race is likely to be bias in law enforcement against innocents of that race; (2) AI-empowered policing, the primary source of observed bias against a race is likely to be bias in law enforcement against criminals of that race; and (3) police-civilian interaction, the primary source of observed bias against a race could be bias in law enforcement against that race or bias from the general public in reporting against the other race. Through an extensive empirical study using police-civilian interaction data and 911 call data, we find an instance of such a counter-intuitive phenomenon: in New Orleans, the observed bias is against the majority race and the likely reason for it is the over-reporting (via 911 calls) of incidents involving the minority race by the general public.

翻译：我们旨在开发一种数据驱动的方法来评估执法系统中由种族引发的偏见。尽管近期研究已利用警方拦截数据探讨警察与平民互动中的这一问题，但仍存在两个关键局限：其一，只有在控制真实犯罪率与种族因素的前提下，才能准确定量偏差，而先前研究缺乏对犯罪率的考量；其二，执法系统具有多阶段特性，因此需在"因果互动链"中隔离偏差的真实来源，而非仅关注最终结果——这有助于指导改革。本研究通过引入犯罪率的多阶段因果框架应对上述挑战。我们提供理论刻画及配套数据驱动方法，用于评估：(a) 是否存在任何形式的种族偏见，以及 (b) 若存在，这种偏见的主要来源（基于种族与犯罪率维度）。本框架识别出三种具有不同特征的标准情景：在 (1) 机场安检等场景中，针对某一种族观察到的主要偏差来源，很可能源于执法部门对该种族无辜者的偏见；(2) AI赋能警务场景中，针对某一种族观察到的主要偏差来源，很可能源于执法部门对该种族犯罪者的偏见；(3) 警察-平民互动场景中，针对某一种族观察到的主要偏差来源，可能是执法部门对该种族的偏见，或是公众在举报时对另一种族产生的偏见。通过使用警察-平民互动数据与911报警数据开展的大规模实证研究，我们发现了一个反直觉现象：在新奥尔良市，观测到的偏见指向多数种族，而其原因很可能是普通民众通过911报警电话过度举报了涉及少数种族的事件。