Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

翻译：大型语言模型已被证明能够基于少量示例或自然语言指令执行新任务。尽管这些能力已带来广泛采用，但大多数大型语言模型由资源丰富的组织开发，且通常不向公众开放。作为推动这一强大技术民主化进程的一步，我们推出BLOOM，一个由数百名研究者协作设计构建的1760亿参数开源语言模型。BLOOM是一个仅解码器的Transformer语言模型，基于ROOTS语料库训练而成——该数据集包含46种自然语言和13种编程语言（共59种语言）的数百个来源。我们发现BLOOM在各类基准测试中取得了具有竞争力的表现，在经多任务提示微调后性能更为强劲。为促进未来基于大型语言模型的研究与应用，我们根据《负责任人工智能许可协议》公开发布模型和代码。