Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM) agents. However, most environments are scored only by game outcomes such as win rates and largely remain to text-only interaction, making it difficult to tell whether an agent's language is actually grounded in what it perceived and did, or to identify the failure modes underlying its behavior. To address this gap, we introduce QUACK, an open-source environment and evaluation framework for auditing the grounding of agent language in multimodal social reasoning. QUACK evaluates agents at three levels: game outcomes, behavioral trajectories, and utterance-level consistency. Its core Statement Verification Pipeline reconstructs each agent's ground-truth trajectory from engine logs and checks every discussion claim against it, automatically flagging spatial hallucination, unsupported accusation, deception collapse, and language-action inconsistency. Evaluating three frontier VLMs in both homogeneous and cross-model adversarial settings, we find that even the strongest agent hallucinates 15.1% of its verifiable spatial claims and makes over half of its accusations without grounded evidence. We release the full engine, evaluation framework, toolkit, and logs at https://github.com/AAAAA-Academia-Attractions/QUACK.

翻译：摘要：社交推理游戏已成为测试大型语言模型（LLM）智能体在推理、欺骗、协作及信念建模方面能力的流行测试平台。然而，现有环境通常仅通过胜率等游戏结果进行评分，且大部分仍局限于纯文本交互，导致难以判断智能体的语言是否真正基于其感知与行动，也难以识别其行为背后的失败模式。为填补这一空白，我们提出了QUACK——一个用于审计多模态社交推理中智能体语言基础的开源环境与评估框架。QUACK从三个层面评估智能体：游戏结果、行为轨迹以及话语级一致性。其核心的陈述验证流水线从引擎日志重建每个智能体的真实轨迹，并对照每个讨论声明进行核验，自动标记空间幻觉、无依据指控、欺骗崩溃及语言-动作不一致等问题。我们在同质与跨模型对抗设置中对三种前沿视觉语言模型进行评估后发现，即使是最强的智能体，其可验证的空间声明中仍有15.1%存在幻觉，且超过一半的指控缺乏基础证据。我们已在https://github.com/AAAAA-Academia-Attractions/QUACK开源完整引擎、评估框架、工具包及日志。