We are interested in developing a data-driven method to evaluate race-induced biases in law enforcement systems. While the recent works have addressed this question in the context of police-civilian interactions using police stop data, they have two key limitations. First, bias can only be properly quantified if true criminality is accounted for in addition to race, but it is absent in prior works. Second, law enforcement systems are multi-stage and hence it is important to isolate the true source of bias within the "causal chain of interactions" rather than simply focusing on the end outcome; this can help guide reforms. In this work, we address these challenges by presenting a multi-stage causal framework incorporating criminality. We provide a theoretical characterization and an associated data-driven method to evaluate (a) the presence of any form of racial bias, and (b) if so, the primary source of such a bias in terms of race and criminality. Our framework identifies three canonical scenarios with distinct characteristics: in settings like (1) airport security, the primary source of observed bias against a race is likely to be bias in law enforcement against innocents of that race; (2) AI-empowered policing, the primary source of observed bias against a race is likely to be bias in law enforcement against criminals of that race; and (3) police-civilian interaction, the primary source of observed bias against a race could be bias in law enforcement against that race or bias from the general public in reporting against the other race. Through an extensive empirical study using police-civilian interaction data and 911 call data, we find an instance of such a counter-intuitive phenomenon: in New Orleans, the observed bias is against the majority race and the likely reason for it is the over-reporting (via 911 calls) of incidents involving the minority race by the general public.

翻译：我们致力于开发一种数据驱动的方法来评估执法系统中由种族引发的偏见。尽管近期研究已利用警方拦截数据探讨警察与平民互动中的这一问题，但仍存在两个关键局限。其一，偏见需在同时考量真实犯罪率与种族因素时方能准确定量，而先前研究未纳入犯罪率。其二，执法系统具有多阶段特征，因此需在“互动的因果链条”中隔离偏见的真实源头，而非仅关注最终结果——这有助于指导改革。本研究通过提出一个包含犯罪率的多阶段因果框架来应对上述挑战。我们从理论上刻画了该框架，并开发了相应的数据驱动方法，用于评估：(a) 是否存在任何形式的种族偏见；(b) 若存在，该偏见的主要来源（基于种族与犯罪率维度）。该框架识别出三种具有差异化特征的典型场景：(1) 机场安检场景中，针对某一种族所观察到的偏见主要源于执法机关对该种族无辜者的偏见；(2) AI赋能警务场景中，针对某一种族所观察到的偏见主要源于执法机关对该种族犯罪者的偏见；(3) 警察-平民互动场景中，针对某一种族所观察到的偏见可能源于执法机关对该种族的偏见，或公众在举报中对另一种族存在的偏见。通过使用警察-平民互动数据与911报警电话数据进行广泛实证研究，我们发现了一个反直觉现象：在新奥尔良市，观察到的偏见反而针对多数种族，其可能原因是公众通过911电话过度举报涉及少数种族的事件。