Recent legislation required AI platforms to provide APIs for regulators to assess their compliance with the law. Research has nevertheless shown that platforms can manipulate their API answers through fairwashing. Facing this threat for reliable auditing, this paper studies the benefits of the joint use of platform scraping and of APIs. In this setup, we elaborate on the use of scraping to detect manipulated answers: since fairwashing only manipulates API answers, exploiting scraps may reveal a manipulation. To abstract the wide range of specific API-scrap situations, we introduce a notion of proxy that captures the consistency an auditor might expect between both data sources. If the regulator has a good proxy of the consistency, then she can easily detect manipulation and even bypass the API to conduct her audit. On the other hand, without a good proxy, relying on the API is necessary, and the auditor cannot defend against fairwashing. We then simulate practical scenarios in which the auditor may mostly rely on the API to conveniently conduct the audit task, while maintaining her chances to detect a potential manipulation. To highlight the tension between the audit task and the API fairwashing detection task, we identify Pareto-optimal strategies in a practical audit scenario. We believe this research sets the stage for reliable audits in practical and manipulation-prone setups.

翻译：近期立法要求AI平台提供应用程序编程接口（API），以便监管机构评估其法律合规性。然而研究表明，平台可能通过"公平清洗"操纵API响应。面对这一可靠审计的威胁，本文研究了联合使用平台抓取与API的优势。在此框架下，我们阐述了利用抓取技术检测被操纵响应的原理：由于公平清洗仅操纵API响应，抓取数据可能揭示操纵行为。为抽象化各类特定API-抓取场景，我们提出了代理概念，用以刻画审计方对两类数据源一致性的合理预期。若监管机构掌握良好的一致性代理，不仅能轻易检测操纵行为，甚至可绕过API直接开展审计。反之，若缺乏良好代理，审计方将不得不依赖API，且无法抵御公平清洗。在此基础上，我们模拟了审计方在主要依赖API便捷执行审计任务的同时，仍保留检测潜在操纵可能性的实践场景。为突出审计任务与API公平清洗检测任务之间的张力，我们在实际审计场景中识别出帕累托最优策略。我们相信本研究为在实用且易受操纵的环境中进行可靠审计奠定了基础。