The European Union has proposed the Artificial Intelligence Act intending to regulate AI systems, especially those used in high-risk, safety-critical applications such as healthcare. Among the Act's articles are detailed requirements for transparency and explainability. The field of explainable AI (XAI) offers technologies that could address many of these requirements. However, there are significant differences between the solutions offered by XAI and the requirements of the AI Act, for instance, the lack of an explicit definition of transparency. We argue that collaboration is essential between lawyers and XAI researchers to address these differences. To establish common ground, we give an overview of XAI and its legal relevance followed by a reading of the transparency and explainability requirements of the AI Act and the related General Data Protection Regulation (GDPR). We then discuss four main topics where the differences could induce issues. Specifically, the legal status of XAI, the lack of a definition of transparency, issues around conformity assessments, and the use of XAI for dataset-related transparency. We hope that increased clarity will promote interdisciplinary research between the law and XAI and support the creation of a sustainable regulation that fosters responsible innovation.

翻译：欧盟提出了《人工智能法案》，旨在规范人工智能系统，特别是那些用于医疗保健等高风险、安全关键领域的系统。该法案的条款包含了对透明度和可解释性的详细要求。可解释人工智能（XAI）领域提供了能够满足其中许多要求的技术。然而，XAI提供的解决方案与《人工智能法案》的要求之间存在显著差异，例如缺乏对透明度的明确定义。我们认为，律师和XAI研究人员之间的合作对于解决这些差异至关重要。为建立共同基础，我们概述了XAI及其法律相关性，随后解读了《人工智能法案》以及相关《通用数据保护条例》（GDPR）中的透明度和可解释性要求。接着，我们讨论了四个可能因这些差异引发问题的主要议题，具体包括：XAI的法律地位、透明度的定义缺失、合格评定相关的问题，以及利用XAI实现数据集相关的透明度。我们希望，更清晰的阐述将促进法律与XAI之间的跨学科研究，并支持创建可持续的法规，从而推动负责任的创新。