As Natural Language Processing (NLP) systems are increasingly employed in intricate social environments, a pressing query emerges: Can these NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models (LLMs)? This paper probes the collaboration mechanisms among contemporary NLP systems by melding practical experiments with theoretical insights. We fabricate four unique `societies' comprised of LLM agents, where each agent is characterized by a specific `trait' (easy-going or overconfident) and engages in collaboration with a distinct `thinking pattern' (debate or reflection). Evaluating these multi-agent societies on three benchmark datasets, we discern that LLM agents navigate tasks by leveraging diverse social behaviors, from active debates to introspective reflections. Notably, certain collaborative strategies only optimize efficiency (using fewer API tokens), but also outshine previous top-tier approaches. Moreover, our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity or majority rule, mirroring foundational Social Psychology theories. In conclusion, we integrate insights from Social Psychology to contextualize the collaboration of LLM agents, inspiring further investigations into the collaboration mechanism for LLMs. We commit to sharing our code and datasets (already submitted in supplementary materials), hoping to catalyze further research in this promising avenue (All code and data are available at \url{https://github.com/zjunlp/MachineSoM}.).

翻译：随着自然语言处理（NLP）系统日益应用于复杂社会环境中，一个迫切问题随之浮现：在由多个大语言模型（LLM）构成的智能体社会中，这些NLP系统能否展现类似人类的协作智能？本文通过融合实践实验与理论洞见，深入探究当代NLP系统间的协作机制。我们构建了四个由LLM智能体组成的独特"社会"模型，每个智能体具有特定"特质"（随和型或过度自信型），并采用不同的"思维模式"（辩论或反思）进行协作。在三个基准数据集上评估这些多智能体社会后，我们发现LLM智能体通过利用从主动辩论到内省反思的多样化社会行为来完成任务。值得注意的是，某些协作策略不仅优化了效率（使用更少的API令牌），更超越了先前的最优方法。此外，我们的研究结果进一步表明，LLM智能体展现出从众效应、多数规则等类人社会行为，这与社会心理学基础理论相呼应。最后，我们整合社会心理学洞见来诠释LLM智能体的协作机制，冀望激发对LLM协作机制的进一步探索。我们将公开代码与数据集（已提交至补充材料），期待助推这一前沿方向的深入研究（所有代码与数据详见\url{https://github.com/zjunlp/MachineSoM}）。