Emergent misalignment (EM) occurs when narrow finetuning causes a model to behave dangerously outside the finetuning task. Standard training signals can miss this shift, making reliable detection costly if it depends on repeated behavioral evaluation. We ask whether emergent misalignment can instead be detected from internal representations during finetuning. Using seven alignment-relevant traits encoded as linear directions in activation space, we track representational drift across training checkpoints in four open-source 7-9B LLMs. EM-relevant drift concentrates on a low-dimensional axis that explains 65.5% of the variance, revealing a geometric signature in the studied regime. A low-overhead monitor built on this drift profile detects dangerous checkpoints with 2.2% false negative rate, 2.9% false positive rate, and 0.990 AUROC on held-out perturbation types, outperforming unsupervised PCA and SAE baselines. Stress tests on two 14B models, longer finetuning runs, and misaligned starting points identify key deployment boundaries. These results position trait-space monitoring as a practical complement to behavioral evaluation for EM detection during LoRA-based finetuning, while showing that deployment across substantially different regimes may require recalibration.
翻译:涌现性失配(EM)指当窄域微调导致模型在微调任务之外表现出危险行为。标准训练信号可能遗漏这种偏移,使得依赖重复行为评估的可靠检测代价高昂。我们探究能否从微调过程中的内部表征检测涌现性失配。通过将七种对齐相关特征编码为激活空间中的线性方向,我们追踪四个开源7-9B大语言模型训练检查点上的表征漂移。EM相关漂移集中于一个低维轴,该轴解释了65.5%的方差,揭示了所研究范式中的几何特征。基于该漂移特征构建的低开销监测器可检测危险检查点,在留出扰动类型上实现2.2%假阴性率、2.9%假阳性率及0.990 AUROC,优于无监督主成分分析和稀疏自编码器基线。在14B模型、更长微调运行及错位起始点上的压力测试确定了关键部署边界。这些结果将特征空间监测定位为LoRA微调中涌现性失配检测的行为评估实用补充方案,同时表明跨显著不同范式部署可能需要重新校准。