Agents built with large language models (LLMs) have recently achieved great advancements. However, most of the efforts focus on single-agent or cooperative settings, leaving more general multi-agent environments underexplored. We propose a new framework powered by reinforcement learning (RL) to develop strategic language agents, i.e., LLM-based agents with strategic thinking ability, for a popular language game, Werewolf. Werewolf is a social deduction game with hidden roles that involves both cooperation and competition and emphasizes deceptive communication and diverse gameplay. Our agent tackles this game by first using LLMs to reason about potential deceptions and generate a set of strategically diverse actions. Then an RL policy, which selects an action from the candidates, is learned by population-based training to enhance the agents' decision-making ability. By combining LLMs with the RL policy, our agent produces a variety of emergent strategies, achieves the highest win rate against other LLM-based agents, and stays robust against adversarial human players in the Werewolf game.

翻译：近期，基于大型语言模型（LLM）构建的智能体取得了显著进展。然而，现有研究多聚焦于单智能体或合作场景，对更普适的多智能体环境的探索仍显不足。针对流行语言类游戏"狼人杀"，我们提出一种融合强化学习（RL）的创新框架，用于开发具备战略思维能力的语言智能体（即基于LLM的智能体）。作为一款包含隐藏角色的社交推理游戏，狼人杀融合了合作与竞争双重机制，尤其强调欺骗性沟通与多样化博弈。我们的智能体通过两个阶段应对挑战：首先利用LLM推理潜在欺骗行为，生成一组具有战略多样性的动作候选集；其次通过种群训练学习强化学习策略，从候选动作中择优执行，从而提升智能体的决策能力。通过将LLM与强化学习策略相结合，我们的智能体展现出丰富的涌现策略，在与其他基于LLM的智能体的对抗中取得最高胜率，并能在狼人杀游戏中对战人类玩家时保持稳健表现。