Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

翻译：大型语言模型（LLMs）已被证明能够基于少量示例或自然语言指令执行新任务。尽管这些能力已带来广泛应用，但大多数LLMs由资源丰富的组织开发，且常不对公众开放。为推进这一强大技术的民主化，我们提出BLOOM——一个由数百名研究者协作设计构建的176B参数开源语言模型。BLOOM是一个仅解码器的Transformer语言模型，在ROOTS语料库上训练而成，该数据集包含46种自然语言和13种编程语言（共59种语言）的数百个来源。我们发现BLOOM在多种基准测试中展现出具有竞争力的性能，并在经过多任务提示微调后取得更优结果。为促进未来基于LLM的研究与应用，我们依据《责任式AI许可协议》公开发布模型与代码。