The emergence of Large Language Models (LLMs), has opened exciting possibilities for constructing computational simulations designed to replicate human behavior accurately. Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies. However, LLMs are complex statistical learners without straightforward deductive rules, making them prone to unexpected behaviors. Hence, it is crucial to study and pinpoint the key behavioral distinctions between humans and LLM-based agents. In this study, we highlight the limitations of LLMs in simulating human interactions, particularly focusing on LLMs' ability to simulate political debates on topics that are important aspects of people's day-to-day lives and decision-making processes. Our findings indicate a tendency for LLM agents to conform to the model's inherent social biases despite being directed to debate from certain political perspectives. This tendency results in behavioral patterns that seem to deviate from well-established social dynamics among humans. We reinforce these observations using an automatic self-fine-tuning method, which enables us to manipulate the biases within the LLM and demonstrate that agents subsequently align with the altered biases. These results underscore the need for further research to develop methods that help agents overcome these biases, a critical step toward creating more realistic simulations.

翻译：大型语言模型（LLMs）的出现为构建能够准确复现人类行为的计算模拟开辟了令人兴奋的可能性。现有研究表明，基于LLM的智能体在表现上越来越接近人类，这引发了利用这些AI智能体替代人类参与者进行行为研究的兴趣。然而，LLMs是复杂的统计学习器，缺乏直接的演绎规则，容易产生意外行为。因此，研究和厘清人类与基于LLM的智能体之间的关键行为差异至关重要。在本研究中，我们重点揭示了LLMs在模拟人类互动方面的局限性，特别关注LLMs模拟政治辩论的能力，这些辩论主题涉及人们日常生活和决策过程的重要方面。我们的研究结果表明，尽管被要求从特定政治立场进行辩论，LLM智能体仍倾向于遵从模型固有的社会偏见。这种倾向导致其行为模式似乎偏离了人类社会中已确立的社会动态。我们通过一种自动自微调方法强化了这些观察，该方法使我们能够操纵LLM内部的偏见，并证明智能体会随后与调整后的偏见保持一致。这些结果强调需要进一步研究开发帮助智能体克服这些偏见的方法，这是创建更真实模拟的关键一步。