Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have become closed off, gated behind proprietary interfaces, with important details of their training data, architectures, and development undisclosed. Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs. To this end, this technical report details the first release of OLMo, a state-of-the-art, truly Open Language Model and its framework to build and study the science of language modeling. Unlike most prior efforts that have only released model weights and inference code, we release OLMo and the whole framework, including training data and training and evaluation code. We hope this release will empower and strengthen the open research community and inspire a new wave of innovation.

翻译：语言模型（LMs）在自然语言处理研究和商业产品中已变得无处不在。随着其商业重要性的激增，最强大的模型被封闭在专有接口之后，其训练数据、架构和开发的重要细节均未公开。鉴于这些细节对科学地研究这些模型（包括其偏见和潜在风险）至关重要，我们认为研究社区有必要获得强大且真正开放的语言模型。为此，本技术报告详细介绍了OLMo的首次发布——这是一个最先进的、真正开放的语言模型及其构建与研究语言模型科学的框架。与以往大多数仅发布模型权重和推理代码的工作不同，我们发布了OLMo及其整个框架，包括训练数据以及训练和评估代码。我们希望这一发布能够赋权并强化开放研究社区，激发新一轮创新浪潮。