The European Union's Artificial Intelligence Act establishes comprehensive requirements for high-risk AI systems, yet the harmonized standards necessary for demonstrating compliance remain not fully developed. In this paper, we investigate the practical application of the Fraunhofer AI assessment catalogue as a certification framework through a complete self-certification cycle of an AI-based facial emotion recognition system. Beginning with a baseline model that has deficiencies, including inadequate demographic representation and prediction uncertainty, we document an enhancement process guided by AI certification requirements. The enhanced system achieves higher accuracy with improved reliability metrics and comprehensive fairness across demographic groups. We focused our assessment on two of the six Fraunhofer catalogue dimensions, reliability and fairness, the enhanced system successfully satisfies the certification criteria for these examined dimensions. We find that the certification framework provides value as a proactive development tool, driving concrete technical improvements and generating documentation naturally through integration into the development process. However, fundamental gaps separate structured self-certification from legal compliance: harmonized European standards are not fully available, and AI assessment frameworks and catalogues cannot substitute for them on their own. These findings establish the Fraunhofer AI assessment catalogue as a valuable preparatory tool that complements rather than replaces formal compliance requirements at this time.
翻译:欧盟《人工智能法案》为高风险AI系统确立了全面的合规要求,但用于证明合规性的统一标准尚未完全制定。本文通过一个基于AI的面部情绪识别系统的完整自我认证周期,研究了弗劳恩霍夫AI评估目录作为认证框架的实际应用。我们从一个存在缺陷的基线模型出发,其缺陷包括人口统计代表性不足和预测不确定性,并记录了在AI认证要求指导下进行的系统增强过程。增强后的系统实现了更高的准确率,其可靠性指标得到改善,并在各人口统计组间实现了全面的公平性。我们将评估重点放在弗劳恩霍夫目录六个维度中的两个——可靠性与公平性上,增强后的系统成功满足了所考察维度的认证标准。我们发现,该认证框架作为一种主动的开发工具具有价值,它能推动具体的技术改进,并通过融入开发过程自然生成文档。然而,结构化的自我认证与法律合规之间存在根本性差距:欧洲统一标准尚未完全可用,且AI评估框架和目录本身无法替代这些标准。这些发现表明,弗劳恩霍夫AI评估目录目前是一种有价值的预备性工具,它补充而非取代了正式的合规要求。