We present an automated, contrastive evaluation pipeline for auditing the behavioral impact of interventions on large language models. Given a base model $M_1$ and an intervention model $M_2$, our method compares their free-form, multi-token generations across aligned prompt contexts and produces human-readable, statistically validated natural-language hypotheses describing how the models differ, along with recurring themes that summarize patterns across validated hypotheses. We evaluate the approach in synthetic setting by injecting known behavioral changes and showing that the pipeline reliably recovers them. We then apply it to three real-world interventions, reasoning distillation, knowledge editing and unlearning, demonstrating that the method surfaces both intended and unexpected behavioral shifts, distinguishes large from subtle interventions, and does not hallucinate differences when effects are absent or misaligned with the prompt bank. Overall, the pipeline provides a statistically grounded and interpretable tool for post-hoc auditing of intervention-induced changes in model behavior.

翻译：我们提出了一种自动化、对比性的评估流程，用于审计干预对大型语言模型行为的影响。给定一个基础模型 $M_1$ 和一个干预模型 $M_2$，我们的方法比较它们在对齐提示上下文中的自由形式、多标记生成，并生成可读性强、经统计验证的自然语言假设，描述模型之间的差异，同时提炼出跨验证假设的重复主题。我们在合成场景中评估该方法，通过注入已知行为变化，证明该流程能够可靠地恢复这些变化。随后，我们将该方法应用于三种真实世界干预：推理蒸馏、知识编辑和遗忘，结果表明，该方法能够揭示预期与意外的行为转变，区分较大和细微的干预，并且在效应不存在或与提示库不匹配时不会虚构差异。总体而言，该流程为干预引起的模型行为变化的事后审计提供了一个基于统计且可解释的工具。