Instruction tuning is widely recognized as a key technique for building generalist language models, which has attracted the attention of researchers and the public with the release of InstructGPT~\citep{ouyang2022training} and ChatGPT\footnote{\url{https://chat.openai.com/}}. Despite impressive progress in English-oriented large-scale language models (LLMs), it is still under-explored whether English-based foundation LLMs can perform similarly on multilingual tasks compared to English tasks with well-designed instruction tuning and how we can construct the corpora needed for the tuning. To remedy this gap, we propose the project as an attempt to create a Chinese instruction dataset by various methods adapted to the intrinsic characteristics of 4 sub-tasks. We collect around 200k Chinese instruction tuning samples, which have been manually checked to guarantee high quality. We also summarize the existing English and Chinese instruction corpora and briefly describe some potential applications of the newly constructed Chinese instruction corpora. The resulting \textbf{C}hinese \textbf{O}pen \textbf{I}nstruction \textbf{G}eneralist (\textbf{COIG}) corpora are available in Huggingface\footnote{\url{https://huggingface.co/datasets/BAAI/COIG}} and Github\footnote{\url{https://github.com/BAAI-Zlab/COIG}}, and will be continuously updated.
翻译:指令微调被广泛认为是构建通用语言模型的关键技术,随着InstructGPT~\citep{ouyang2022training}和ChatGPT\footnote{\url{https://chat.openai.com/}}的发布,这一技术已吸引了研究者和公众的关注。尽管面向英语的大规模语言模型(LLMs)取得了显著进展,但基于英语的基础LLMs能否通过精心设计的指令微调在多语言任务中达到与英语任务相当的性能,以及如何构建微调所需的语料库,这两个问题仍有待探索。为弥补这一不足,我们提出该项目,尝试根据4个子任务的内在特性,采用多种方法构建中文指令数据集。我们收集了约20万条经过人工核验的高质量中文指令微调样本。此外,我们总结了现有的英文和中文指令语料库,并简要描述了新构建的中文指令语料库的潜在应用。所得到的**中**文**开**放**指**令**通**用模型(COIG)语料库已在Huggingface\footnote{\url{https://huggingface.co/datasets/BAAI/COIG}}和Github\footnote{\url{https://github.com/BAAI-Zlab/COIG}}上发布,并将持续更新。