AI compliance is becoming increasingly critical as AI systems grow more powerful and pervasive. Yet the rapid expansion of AI policies creates substantial burdens for resource-constrained practitioners lacking policy expertise. Existing approaches typically address one policy at a time, making multi-policy compliance costly. We present PASTA, a scalable compliance tool integrating four innovations: (1) a comprehensive model-card format supporting descriptive inputs across development stages; (2) a policy normalization scheme; (3) an efficient LLM-powered pairwise evaluation engine with cost-saving strategies; and (4) an interface delivering interpretable evaluations via compliance heatmaps and actionable recommendations. Expert evaluation shows PASTA's judgments closely align with human experts ($ρ\geq .626$). The system evaluates five major policies in under two minutes at approximately \$3. A user study (N = 12) confirms practitioners found outputs easy-to-understand and actionable, introducing a novel framework for scalable automated AI governance.
翻译:随着AI系统日益强大和普及,AI合规性正变得愈发关键。然而,AI政策的快速扩张给缺乏政策专业知识且资源受限的实践者带来了沉重负担。现有方法通常一次仅处理单一政策,导致多政策合规成本高昂。本文提出PASTA,一种可扩展的合规性工具,其整合了四项创新:(1) 支持跨开发阶段描述性输入的综合性模型卡片格式;(2) 政策规范化方案;(3) 采用成本节约策略的高效LLM驱动配对评估引擎;(4) 通过合规性热力图与可操作建议提供可解释评估的交互界面。专家评估表明,PASTA的判断与人类专家高度一致($ρ\geq .626$)。该系统可在约两分钟内以3美元左右的成本完成对五项主要政策的评估。一项用户研究(N = 12)证实实践者认为其输出易于理解且具备可操作性,从而为可扩展的自动化AI治理引入了新颖的框架。