As Natural Language Processing (NLP) systems are increasingly employed in intricate social environments, a pressing query emerges: Can these NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models (LLMs)? This paper probes the collaboration mechanisms among contemporary NLP systems by melding practical experiments with theoretical insights. We fabricate four unique `societies' comprised of LLM agents, where each agent is characterized by a specific `trait' (easy-going or overconfident) and engages in collaboration with a distinct `thinking pattern' (debate or reflection). Through evaluating these multi-agent societies on three benchmark datasets, we discern that certain collaborative strategies not only outshine previous top-tier approaches, but also optimize efficiency (using fewer API tokens). Moreover, our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring foundational social psychology theories. In conclusion, we integrate insights from social psychology to contextualize the collaboration of LLM agents, inspiring further investigations into the collaboration mechanism for LLMs. We commit to sharing our code and datasets\footnote{\url{https://github.com/zjunlp/MachineSoM}.}, hoping to catalyze further research in this promising avenue.

翻译：随着自然语言处理系统日益应用于复杂的社会环境，一个紧迫的问题浮现：由多个大语言模型（LLM）构成的智能体社会中，这些NLP系统能否展现出类似人类的协作智能？本文通过融合实践实验与理论洞见，深入探究了当代NLP系统间的协作机制。我们构建了四个由LLM智能体组成的独特"社会"，每个智能体具有特定的"特质"（随和型或过度自信型），并采用不同的"思维模式"（辩论或反思）进行协作。通过在三组基准数据集上评估这些多智能体社会，我们发现某些协作策略不仅优于先前的最优方法，还能优化效率（使用更少的API令牌）。此外，我们的结果进一步表明，LLM智能体表现出类似人类的社会行为，如从众和共识达成，这与社会心理学基础理论相呼应。最后，我们整合社会心理学视角，对LLM智能体的协作机制进行情境化分析，以激发对LLM协作机制的进一步探索。我们将公开代码与数据集\footnote{\url{https://github.com/zjunlp/MachineSoM}.}，以期推动这一前景广阔的研究方向。