We show how to optimally regulate prediction algorithms in a world where an agent uses complex 'black-box' prediction functions to make decisions such as lending, medical testing, or hiring, and where a principal is limited in how much she can learn about the agent's black-box model. We show that limiting agents to prediction functions that are simple enough to be fully transparent is inefficient as long as the misalignment is limited and first-best prediction functions are sufficiently complex. Algorithmic audits can improve welfare, but the gains depend on the design of the audit tools. Tools that focus on minimizing overall information loss, the focus of many explainer tools, will generally be inefficient since they focus on explaining the average behavior of the prediction function. Targeted tools that focus on the source of incentive misalignment, e.g., excess false positives or racial disparities, can provide second-best solutions. We provide empirical support for our theoretical findings using an application in consumer lending, where we document that complex models regulated based on context-specific explanation tools outperform simple, fully transparent models. This gain from complex models represents a Pareto improvement across our empirical applications that are preferred both by the lender and from the perspective of the financial regulator.

翻译：我们展示了如何在代理人使用复杂“黑箱”预测函数进行贷款、医疗检测或雇佣等决策，且委托人对其黑箱模型的了解有限的情况下，最优地规制预测算法。研究表明，只要激励偏差有限且最优预测函数足够复杂，将代理人限制于完全透明的简单预测函数便是低效的。算法审计可以改善社会福利，但收益取决于审计工具的设计。许多解释工具侧重于最小化整体信息损失，此类工具通常效率低下，因其仅着眼于解释预测函数的平均行为。而针对激励偏差根源（如过度假阳性或种族差异）的靶向工具，则可提供次优解决方案。我们通过消费贷款应用为理论发现提供了实证支持：经情境特异性解释工具规制的复杂模型显著优于完全透明的简单模型。这种复杂模型的增效在实证应用中体现为帕累托改进——其既受贷款方青睐，亦符合金融监管机构的视角。