Current language model evaluations measure what models know under ideal conditions but not how robustly they know it under realistic stress. Static benchmarks like MMLU and TruthfulQA cannot distinguish a model that lacks knowledge from one whose verification mechanisms collapse when information degrades or adversaries probe for weaknesses. We introduce the Drill-Down and Fabricate Test (DDFT), a protocol that measures epistemic robustness: a model's ability to maintain factual accuracy under progressive semantic compression and adversarial fabrication. We propose a two-system cognitive model comprising a Semantic System that generates fluent text and an Epistemic Verifier that validates factual accuracy. Our findings, based on evaluating 9 frontier models across 8 knowledge domains at 5 compression levels (1,800 turn-level evaluations), reveal that epistemic robustness is orthogonal to conventional design paradigms. Neither parameter count (r=0.083, p=0.832) nor architectural type (r=0.153, p=0.695) significantly predicts robustness, suggesting it emerges from training methodology and verification mechanisms distinct from current approaches. Error detection capability strongly predicts overall robustness (rho=-0.817, p=0.007), indicating this is the critical bottleneck. We find that flagship models exhibit brittleness despite their scale, while smaller models can achieve robust performance, challenging assumptions about the relationship between model size and reliability. The DDFT framework provides both theoretical foundation and practical tools for assessing epistemic robustness before deployment in critical applications.

翻译：当前的语言模型评估衡量的是模型在理想条件下的知识掌握情况，而非其在现实压力下知识掌握的鲁棒性。静态基准测试（如MMLU和TruthfulQA）无法区分一个缺乏知识的模型与一个在信息质量下降或对手探测弱点时其验证机制崩溃的模型。我们提出了钻探与伪造测试（DDFT），这是一种测量认知鲁棒性的协议：即模型在渐进式语义压缩和对抗性伪造条件下保持事实准确性的能力。我们提出了一个包含两个系统的认知模型：一个生成流畅文本的语义系统和一个验证事实准确性的认知验证器。我们的发现基于在8个知识领域、5个压缩级别下评估9个前沿模型（共1,800轮次评估），结果表明认知鲁棒性与传统设计范式正交。无论是参数量（r=0.083, p=0.832）还是架构类型（r=0.153, p=0.695）均无法显著预测鲁棒性，这表明它源于与当前方法不同的训练方法和验证机制。错误检测能力能强有力地预测整体鲁棒性（rho=-0.817, p=0.007），表明这是关键瓶颈。我们发现，尽管旗舰模型规模庞大，却表现出脆弱性，而较小的模型可以实现鲁棒的性能，这对关于模型规模与可靠性之间关系的假设提出了挑战。DDFT框架为在关键应用部署前评估认知鲁棒性提供了理论基础和实用工具。