Studies of reinforcement learning in humans and animals have demonstrated a preference for options that yielded relatively better outcomes in the past, even when those options are associated with lower absolute reward. The present study tested whether large language models would exhibit a similar bias. We had gpt-4-1106-preview (GPT-4 Turbo) and Llama-2-70B make repeated choices between pairs of options with the goal of maximizing payoffs. A complete record of previous outcomes was included in each prompt. Both models exhibited relative value decision biases similar to those observed in humans and animals. Making relative comparisons among outcomes more explicit magnified the bias, whereas prompting the models to estimate expected outcomes caused the bias to disappear. These results have implications for the potential mechanisms that contribute to context-dependent choice in human agents.

翻译：人类与动物的强化学习研究表明，即使某些选项与较低的绝对奖励相关，个体仍会倾向于选择过去带来相对更好结果的选项。本研究测试了大型语言模型是否表现出类似偏差。我们让gpt-4-1106-preview（GPT-4 Turbo）和Llama-2-70B在成对选项中反复选择，以最大化收益为目标，每个提示词均包含完整的历史结果记录。两个模型均表现出与人类和动物相似的相对价值决策偏差。将结果之间的相对比较明确化会放大这种偏差，而提示模型估计预期结果则使偏差消失。这些结果对理解人类决策中情境依赖选择的潜在机制具有启示意义。