Activation monitors-lightweight probes trained on a language model's internal representations-are an increasingly common layer in deployment safety stacks. Deployed models however are rarely static: they are quantized, fine-tuned, adapted with LoRA, or served with merged adapters while the monitor remains frozen. We present the first systematic test of whether this implicit contract holds: whether activation monitors trained on a base model remain reliable after these routine model updates. Across multiple safety-relevant monitors, model depths, update families, and open-weight models, we find a sharp split: quantization-style updates largely preserve frozen probe performance, while fine-tuning-style updates frequently make probes stale. Fragility is highly monitor-dependent, with privacy/PII probes most affected and refusal-compliance probes comparatively stable, showing that retraining a behavior need not stale its corresponding monitor. QLoRA is especially damaging despite NF4 quantization alone being relatively benign, suggesting that quantization becomes riskier when combined with adaptation. We further show that degradation is predictable from pre-deployment features, enabling revalidation budgets to be triaged toward the monitors most likely to fail. These results suggest that fine-tuning should trigger activation-monitor revalidation by default, while prediction can help prioritize which monitors to check first.
翻译:激活监控器——一种基于语言模型内部表示训练的轻量级探针——正日益成为部署安全栈中的常见组件。然而,已部署的模型很少是静态的:它们会经历量化、微调、通过LoRA适配,或与合并适配器一同提供服务,而监控器则保持冻结。我们首次系统性地检验了这种隐含契约是否成立:即,在这些常规模型更新后,基于基础模型训练的激活监控器是否仍能保持可靠。针对多种安全相关监控器、模型深度、更新类型及开放权重模型,我们发现了一个显著的分化:量化类更新基本保持了冻结探针的性能,而微调类更新则常导致探针陈旧。脆弱性高度依赖于具体监控器,其中隐私/个人身份信息(PII)探针受影响最大,而拒绝-遵从性监控器相对稳定,这表明重新训练某个行为并不必然导致其对应监控器失效。QLoRA尤其具有破坏性,尽管单独的NF4量化相对无害,这表明量化在与适配结合时风险会增大。我们进一步证明,这种退化可从部署前的特征进行预测,从而能够将重新验证的预算优先分配给最可能失效的监控器。这些结果表明,微调应默认触发激活监控器的重新验证,而预测则有助于优先确定应首先检查哪些监控器。