The widespread use of generative AI systems is coupled with significant ethical and social challenges. As a result, policymakers, academic researchers, and social advocacy groups have all called for such systems to be audited. However, existing auditing procedures fail to address the governance challenges posed by generative AI systems, which display emergent capabilities and are adaptable to a wide range of downstream tasks. In this chapter, we address that gap by outlining a novel blueprint for how to audit such systems. Specifically, we propose a three-layered approach, whereby governance audits (of technology providers that design and disseminate generative AI systems), model audits (of generative AI systems after pre-training but prior to their release), and application audits (of applications based on top of generative AI systems) complement and inform each other. We show how audits on these three levels, when conducted in a structured and coordinated manner, can be a feasible and effective mechanism for identifying and managing some of the ethical and social risks posed by generative AI systems. That said, it is important to remain realistic about what auditing can reasonably be expected to achieve. For this reason, the chapter also discusses the limitations not only of our three-layered approach but also of the prospect of auditing generative AI systems at all. Ultimately, this chapter seeks to expand the methodological toolkit available to technology providers and policymakers who wish to analyse and evaluate generative AI systems from technical, ethical, and legal perspectives.

翻译：生成式人工智能系统的广泛应用伴随着重大的伦理与社会挑战。因此，政策制定者、学术研究机构和社会倡导团体均呼吁对此类系统实施审计。然而，现有审计程序未能应对生成式人工智能系统所引发的治理挑战——这类系统展现出涌现能力，且能适应广泛的下游任务。本章通过提出一种新型审计蓝图来填补这一空白。具体而言，我们提出一种三层审计框架：治理层审计（针对设计和传播生成式人工智能系统的技术提供商）、模型层审计（针对预训练完成后但发布前的生成式人工智能系统）以及应用层审计（针对基于生成式人工智能系统构建的应用程序）。这三个层面的审计相互补充、相互印证。我们论证了当这三个层面的审计以结构化、协调化的方式实施时，能够成为识别和管理生成式人工智能系统部分伦理与社会风险的可行且有效的机制。尽管如此，仍需对审计可实现的合理预期保持现实态度。为此，本章不仅讨论了我们提出的三层审计框架的局限性，也探讨了生成式人工智能系统审计本身面临的根本性限制。最终，本章旨在为希望从技术、伦理和法律角度分析与评估生成式人工智能系统的技术提供商和政策制定者，拓展其可用的方法论工具箱。