As AI systems evolve from isolated predictors into complex, heterogeneous ecosystems of foundation models and specialized adapters, distinguishing genuine behavioral novelty from functional redundancy becomes a critical governance challenge. Here, we introduce a statistical framework for auditing model uniqueness based on In-Silico Quasi-Experimental Design (ISQED). By enforcing matched interventions across models, we isolate intrinsic model identity and quantify uniqueness as the Peer-Inexpressible Residual (PIER), i.e. the component of a target's behavior strictly irreducible to any stochastic convex combination of its peers, with vanishing PIER characterizing when such a routing-based substitution becomes possible. We establish the theoretical foundations of ecosystem auditing through three key contributions. First, we prove a fundamental limitation of observational logs: uniqueness is mathematically non-identifiable without intervention control. Second, we derive a scaling law for active auditing, showing that our adaptive query protocol achieves minimax-optimal sample efficiency ($dσ^2γ^{-2}\log(Nd/δ)$). Third, we demonstrate that cooperative game-theoretic methods, such as Shapley values, fundamentally fail to detect redundancy. We implement this framework via the DISCO (Design-Integrated Synthetic Control) estimator and deploy it across diverse ecosystems, including computer vision models (ResNet/ConvNeXt/ViT), large language models (BERT/RoBERTa), and city-scale traffic forecasters. These results move trustworthy AI beyond explaining single models: they establish a principled, intervention-based science of auditing and governing heterogeneous model ecosystems.
翻译:随着人工智能系统从孤立的预测器演变为由基础模型与专用适配器构成的复杂异构生态系统,区分真正的行为新颖性与功能冗余已成为关键治理挑战。本文提出一种基于**计算模拟准实验设计**的统计框架,用于审计模型独特性。通过跨模型实施匹配干预,我们分离出模型的内在身份,并将独特性量化为**同伴不可表达残差**——即目标模型行为中严格无法通过其同行的任何随机凸组合还原的组成部分;当PIER趋近于零时,则表征基于路由的替代成为可能。我们通过三项核心贡献建立了生态系统审计的理论基础:首先,我们证明观测日志存在根本局限——若无干预控制,独特性在数学上不可识别。其次,我们推导出主动审计的缩放定律,表明自适应查询协议可实现极小极大最优样本效率。第三,我们论证合作博弈论方法无法检测冗余。我们通过**设计集成合成控制**估计器实现该框架,并将其部署于多样化生态系统,包括计算机视觉模型、大规模语言模型及城市级交通预测系统。这些成果将可信人工智能的研究范畴从解释单一模型,推进至建立基于干预原则的异构模型生态系统审计与治理科学体系。