The remarkable success of Large Language Models (LLMs) has ushered natural language processing (NLP) research into a new era. Despite their diverse capabilities, LLMs trained on different corpora exhibit varying strengths and weaknesses, leading to challenges in maximizing their overall efficiency and versatility. To address these challenges, recent studies have explored collaborative strategies for LLMs. This paper provides a comprehensive overview of this emerging research area, highlighting the motivation behind such collaborations. Specifically, we categorize collaborative strategies into three primary approaches: Merging, Ensemble, and Cooperation. Merging involves integrating multiple LLMs in the parameter space. Ensemble combines the outputs of various LLMs. Cooperation} leverages different LLMs to allow full play to their diverse capabilities for specific tasks. We provide in-depth introductions to these methods from different perspectives and discuss their potential applications. Additionally, we outline future research directions, hoping this work will catalyze further studies on LLM collaborations and paving the way for advanced NLP applications.

翻译：大型语言模型（LLM）的显著成功将自然语言处理（NLP）研究带入了一个新时代。尽管LLM具备多样化的能力，但基于不同语料库训练的模型表现出各异的优势与不足，这为最大化其整体效率与通用性带来了挑战。为应对这些挑战，近期研究探索了LLM的协同策略。本文对这一新兴研究领域进行了全面综述，重点阐述了此类协同背后的动机。具体而言，我们将协同策略归纳为三种主要方法：融合、集成与协作。融合涉及在参数空间中对多个LLM进行整合；集成通过综合不同LLM的输出结果实现协同；协作则通过发挥不同LLM的多样化能力以完成特定任务。我们从多视角深入介绍了这些方法，并探讨了其潜在应用场景。此外，本文还展望了未来研究方向，期望本工作能推动LLM协同研究的进一步发展，并为高级NLP应用开辟道路。