Agents built with large language models (LLMs) have shown great potential across a wide range of domains. However, in complex decision-making tasks, pure LLM-based agents tend to exhibit intrinsic bias in their choice of actions, which is inherited from the model's training data and results in suboptimal performance. To develop strategic language agents, i.e., agents that generate flexible language actions and possess strong decision-making abilities, we propose a novel framework that powers LLM-based agents with reinforcement learning (RL). We consider Werewolf, a popular social deduction game, as a challenging testbed that emphasizes versatile communication and strategic gameplay. To mitigate the intrinsic bias in language actions, our agents use an LLM to perform deductive reasoning and generate a diverse set of action candidates. Then an RL policy trained to optimize the decision-making ability chooses an action from the candidates to play in the game. Extensive experiments show that our agents overcome the intrinsic bias and outperform existing LLM-based agents in the Werewolf game. We also conduct human-agent experiments and find that our agents achieve human-level performance and demonstrate strong strategic play.

翻译：基于大型语言模型（LLM）构建的体已在多个领域展现出巨大潜力。然而，在复杂决策任务中，纯LLM体倾向于在其动作选择中表现出固有偏见，这种偏见源自模型训练数据，导致次优性能。为开发战略性语言体（即能生成灵活语言动作并具备强大决策能力的体），我们提出一种新框架，将强化学习（RL）赋能于LLM体。我们选取狼人杀——一种流行的社交推理游戏——作为强调多面沟通与策略玩法的挑战性测试平台。为缓解语言动作中的固有偏见，我们的体利用LLM进行演绎推理并生成多样的候选动作集。然后，一个为优化决策能力而训练的RL策略从候选动作中选择一个用于游戏。广泛实验表明，我们的体克服了固有偏见，并在狼人杀游戏中优于现有LLM体。我们还进行了人机实验，发现我们的体达到了人类水平的表现，并展现出强大的策略玩法。