Agents built with large language models (LLMs) have recently achieved great advancements. However, most of the efforts focus on single-agent or cooperative settings, leaving more general multi-agent environments underexplored. We propose a new framework powered by reinforcement learning (RL) to develop strategic language agents, i.e., LLM-based agents with strategic thinking ability, for a popular language game, Werewolf. Werewolf is a social deduction game with hidden roles that involves both cooperation and competition and emphasizes deceptive communication and diverse gameplay. Our agent tackles this game by first using LLMs to reason about potential deceptions and generate a set of strategically diverse actions. Then an RL policy, which selects an action from the candidates, is learned by population-based training to enhance the agents' decision-making ability. By combining LLMs with the RL policy, our agent produces a variety of emergent strategies, achieves the highest win rate against other LLM-based agents, and stays robust against adversarial human players in the Werewolf game.

翻译：基于大型语言模型（LLM）构建的智能体近期取得了显著进展。然而，大多数研究集中于单智能体或合作场景，未充分探索更普适的多智能体环境。我们提出了一种由强化学习（RL）驱动的新框架，用于开发面向热门语言游戏"狼人杀"的策略性语言智能体，即具备战略思维能力的LLM基础智能体。狼人杀是一种包含隐藏身份的社会推理游戏，兼具合作与竞争元素，强调欺骗性沟通与多样化玩法。我们的智能体首先利用LLM推理潜在欺骗行为，生成一组具有战略多样性的行动方案。随后通过群体训练学习强化学习策略，从候选方案中选择行动，以增强智能体的决策能力。通过将LLM与强化学习策略相结合，我们的智能体涌现出多种策略，在与基于LLM的其他智能体对战中取得最高胜率，并在狼人杀游戏中保持对人类受试者的鲁棒性。