As machine learning models become increasingly embedded in societal infrastructure, auditing them for bias is of growing importance. However, in real-world deployments, auditing is complicated by the fact that model owners may adaptively update their models in response to changing environments, such as financial markets. These updates can alter the underlying model class while preserving certain properties of interest, raising fundamental questions about what can be reliably audited under such shifts. In this work, we study group fairness auditing under arbitrary updates. We consider general shifts that modify the pre-audit model class while maintaining invariance of the audited property. Our goals are two-fold: (i) to characterize the information complexity of allowable updates, by identifying which strategic changes preserve the property under audit; and (ii) to efficiently estimate auditing properties, such as group fairness, using a minimal number of labeled samples. We propose a generic framework for PAC auditing based on an Empirical Property Optimization (EPO) oracle. For statistical parity, we establish distribution-free auditing bounds characterized by the SP dimension, a novel combinatorial measure that captures the complexity of admissible strategic updates. Finally, we demonstrate that our framework naturally extends to other auditing objectives, including prediction error and robust risk.

翻译：随着机器学习模型日益融入社会基础设施，对其偏见的审计变得愈发重要。然而，在实际部署中，审计工作因模型所有者可能根据环境变化（如金融市场）自适应地更新其模型而变得复杂。这些更新可能在保持某些关键属性的同时改变底层模型类别，从而引发一个根本性问题：在此类变动下，究竟哪些内容能够被可靠地审计？本文研究了任意更新下的群体公平性审计问题。我们考虑了一类广义的变动，这些变动会修改审计前的模型类别，同时保持被审计属性的不变性。我们的目标有两个方面：（i）通过识别哪些策略性变更能够保持被审计的属性，来刻画允许更新的信息复杂度；（ii）利用最少的标注样本，高效地估计审计属性（如群体公平性）。我们提出了一种基于经验属性优化（EPO）预言机的通用PAC审计框架。针对统计奇偶性，我们建立了由SP维度刻画的分布无关审计界，该维度是一种新颖的组合度量，用于捕捉可容许策略性更新的复杂度。最后，我们证明了该框架能够自然地扩展到其他审计目标，包括预测误差和鲁棒风险。