We evaluate Large Language Models (LLMs) in repeated game-theoretic settings to assess whether strategic performance reflects genuine reasoning or reliance on memorized patterns. We consider two canonical games, Prisoner's Dilemma (PD) and Rock-Paper-Scissors (RPS), upon which we introduce counterfactual variants that alter payoff structures and action labels, breaking familiar symmetries and dominance relations. Our multi-metric evaluation framework compares default and counterfactual instantiations, showcasing LLM limitations in incentive sensitivity, structural generalization and strategic reasoning within counterfactual environments.

翻译：我们通过重复博弈论场景评估大语言模型（LLMs），以判断其策略表现源自真实推理还是依赖记忆模式。研究选取两个经典博弈——囚徒困境（PD）与石头-剪刀-布（RPS），并引入反事实变体：通过改变收益结构与行动标签，打破原有对称性与优势关系。我们构建的多维度评估框架通过对比默认设置与反事实变体，揭示了LLMs在反事实环境中存在激励敏感性不足、结构泛化能力薄弱以及策略推理缺陷等问题。