The unparalleled performance of closed-sourced ChatGPT has sparked efforts towards its democratization, with notable strides made by leveraging real user and ChatGPT conversations, as evidenced by Vicuna. However, due to challenges in gathering conversations involving human participation, current endeavors like Baize and UltraChat aim to automatically generate conversational data. They primarily rely on ChatGPT conducting roleplay to simulate human behaviors based on instructions rather than genuine learning from humans, resulting in limited scope, diminished diversity, and an absence of genuine multi-round conversational dynamics. To address the above issues, we target human questions extracted from genuine human-machine conversations as a learning goal and train a user simulator called `Socratic' to produce a high-quality human-centric synthetic conversation dataset. Subsequently, this dataset was used to train our assistant model, named `PlatoLM'. Experimentally, PlatoLM outpaces baseline models in both Vicuna-Bench and MT-Bench by pairwise comparison when considering equivalent training set sizes, and manual evaluation also shows that our model is highly competitive. Impressively, when fine-tuned with the latest LLaMA 2 model, PlatoLM achieves the SOTA performance among 7B models (including LLaMA-2-7B-chat and Vicuna-7B) in MT-Bench benchmark and in Alpaca-Eval benchmark, it ranks second among 7B models, even beating some larger scale models (including LLaMA-2-13B-chat and GPT-3.5). Further in-depth analysis demonstrates the scalability and transferability of our approach. The code is available at https://github.com/FreedomIntelligence/PlatoLM.

翻译：闭源ChatGPT的卓越性能推动了其民主化的进程，通过利用真实用户与ChatGPT对话数据（如Vicuna所证明）已取得显著进展。然而，由于涉及人类参与的对话数据收集存在困难，当前如Baize和UltraChat等研究尝试自动生成对话数据。这些方法主要依赖ChatGPT进行角色扮演，基于指令模拟人类行为而非从人类真实交互中学习，导致数据范围有限、多样性不足且缺乏真实的多轮对话动态。为解决上述问题，我们将从真实人机对话中提取的人类问题作为学习目标，训练了一个名为"Socratic"的用户模拟器，以生成高质量且以人为中心的合成对话数据集。随后，该数据集被用于训练我们的助手模型——"PlatoLM"。实验表明，在同等训练集规模下，PlatoLM在Vicuna-Bench和MT-Bench的成对比较中均优于基准模型，人工评估也显示其具有高度竞争力。令人印象深刻的是，当与最新的LLaMA 2模型进行微调时，PlatoLM在MT-Bench基准测试中取得了7B模型（包括LLaMA-2-7B-chat和Vicuna-7B）的最优性能（SOTA），并在Alpaca-Eval基准测试中位列7B模型第二，甚至超越部分更大规模模型（包括LLaMA-2-13B-chat和GPT-3.5）。进一步深度分析表明，该方法具有良好的可扩展性和可迁移性。代码开源地址：https://github.com/FreedomIntelligence/PlatoLM。