As Natural Language Processing (NLP) systems are increasingly employed in intricate social environments, a pressing query emerges: Can these NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models (LLMs)? This paper probes the collaboration mechanisms among contemporary NLP systems by melding practical experiments with theoretical insights. We fabricate four unique `societies' comprised of LLM agents, where each agent is characterized by a specific `trait' (easy-going or overconfident) and engages in collaboration with a distinct `thinking pattern' (debate or reflection). Through evaluating these multi-agent societies on three benchmark datasets, we discern that certain collaborative strategies not only outshine previous top-tier approaches, but also optimize efficiency (using fewer API tokens). Moreover, our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring foundational social psychology theories. In conclusion, we integrate insights from social psychology to contextualize the collaboration of LLM agents, inspiring further investigations into the collaboration mechanism for LLMs. We commit to sharing our code and datasets\footnote{\url{https://github.com/zjunlp/MachineSoM}.}, hoping to catalyze further research in this promising avenue.

翻译：随着自然语言处理（NLP）系统日益应用于复杂的社会环境中，一个紧迫的问题随之浮现：在由多个大语言模型（LLM）构成的多智能体社会中，这些NLP系统能否模拟类人的协作智能？本文通过融合实验验证与理论洞察，探究了当代NLP系统间的协作机制。我们构建了四个由LLM智能体组成的独特“社会”，其中每个智能体被赋予特定的“特质”（随和型或过度自信型），并以不同的“思维模式”（辩论式或反思式）进行协作。通过在三个基准数据集上评估这些多智能体社会，我们发现某些协作策略不仅超越了以往的顶尖方法，还能提升效率（消耗更少的API令牌）。此外，实验结果进一步表明，LLM智能体展现出类人的社会行为，例如从众倾向与共识达成，这呼应了基础社会心理学理论。最后，我们结合社会心理学视角阐释LLM智能体的协作行为，以启发对大语言模型协作机制的进一步探索。我们承诺公开代码与数据集\footnote{\url{https://github.com/zjunlp/MachineSoM}.}，以期推动这一前沿领域的后续研究。