Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) greatly reduce the required skill and domain knowledge to effectively harness the capabilities of LLMs, increasing their accessibility and utility across various domains. However, state-of-the-art alignment techniques like RLHF rely on high-quality human feedback data, which is expensive to create and often remains proprietary. In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations, a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 complete and fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers. Models trained on OpenAssistant Conversations show consistent improvements on standard benchmarks over respective base models. We release our code and data under a fully permissive licence.

翻译：将大型语言模型（LLMs）与人类偏好对齐已被证明能显著提升其可用性，并推动了如ChatGPT所展示的快速普及。监督微调（SFT）和基于人类反馈的强化学习（RLHF）等对齐技术大幅降低了有效利用LLMs能力所需的技能和领域知识，从而提升了其在各领域的可及性和实用性。然而，像RLHF这样的前沿对齐技术依赖于高质量的人类反馈数据，这类数据创建成本高昂且通常为专有数据。为促进大规模对齐研究的民主化，我们发布了OpenAssistant对话数据集——一个由人类生成并标注的助手风格对话语料库，包含161,443条消息，覆盖35种语言，附有461,292个质量评分，并形成超过10,000个完整且全面标注的对话树。该语料库是全球众包努力的成果，超过13,500名志愿者参与其中。基于OpenAssistant对话语料库训练的模型在标准基准测试上相较各自的基础模型表现出持续改进。我们以完全开放的许可协议发布代码和数据。