Evaluating the strategic reasoning capabilities of Large Language Models (LLMs) requires moving beyond static benchmarks to dynamic, multi-turn interactions. We introduce AIDG (Adversarial Information Deduction Game), a game-theoretic framework that probes the asymmetry between information extraction (active deduction) and information containment (state maintenance) in dialogue. We propose two complementary tasks: AIDG-I, measuring pragmatic strategy in social deduction, and AIDG-II, measuring constraint satisfaction in a structured "20 Questions" setting. Across 439 games with six frontier LLMs, we observe a clear capability asymmetry: models perform substantially better at containment than deduction, with a 350 ELO advantage on defense;(Cohen's d = 5.47). We identify two bottlenecks driving this gap: (1) Information Dynamics, where confirmation strategies are 7.75x more effective than blind deduction (p < 0.00001), and (2) Constraint Adherence, where instruction-following degrades under conversational load, accounting for 41.3% of deductive failures. These findings suggest that while LLMs excel at local defensive coherence, they struggle with the global state tracking required for strategic inquiry.
翻译:评估大型语言模型(LLM)的战略推理能力需要超越静态基准测试,转向动态的多轮交互。我们提出了AIDG(对抗性信息演绎博弈),这是一个博弈论框架,用于探究对话中信息提取(主动演绎)与信息遏制(状态维持)之间的不对称性。我们设计了两个互补的任务:AIDG-I,用于衡量社交演绎中的语用策略;AIDG-II,用于衡量结构化“20个问题”场景中的约束满足能力。通过对六个前沿LLM进行的439场博弈实验,我们观察到一个明显的能力不对称现象:模型在信息遏制方面的表现显著优于信息演绎,在防御任务上具有350 ELO分的优势(Cohen's d = 5.47)。我们识别出导致这一差距的两个关键瓶颈:(1)信息动态性:确认策略的效果是盲目演绎的7.75倍(p < 0.00001);(2)约束遵循性:在对话负载下,模型遵循指令的能力会下降,这导致了41.3%的演绎失败。这些发现表明,虽然LLM在局部防御一致性方面表现出色,但在战略询问所需的全局状态追踪方面仍存在困难。