The use of NLP in the realm of financial technology is broad and complex, with applications ranging from sentiment analysis and named entity recognition to question answering. Large Language Models (LLMs) have been shown to be effective on a variety of tasks; however, no LLM specialized for the financial domain has been reported in literature. In this work, we present BloombergGPT, a 50 billion parameter language model that is trained on a wide range of financial data. We construct a 363 billion token dataset based on Bloomberg's extensive data sources, perhaps the largest domain-specific dataset yet, augmented with 345 billion tokens from general purpose datasets. We validate BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite of internal benchmarks that most accurately reflect our intended usage. Our mixed dataset training leads to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. Additionally, we explain our modeling choices, training process, and evaluation methodology. We release Training Chronicles (Appendix C) detailing our experience in training BloombergGPT.

翻译：自然语言处理（NLP）在金融科技领域的应用广泛而复杂，涉及情感分析、命名实体识别以及问答等任务。大语言模型（LLMs）已被证明在多种任务上有效；然而，文献中尚未报道专门针对金融领域的大语言模型。本研究提出BloombergGPT，一个基于广泛金融数据训练的500亿参数语言模型。我们基于彭博（Bloomberg）海量数据源构建了3630亿词元的数据集——这可能是目前最大的领域专属数据集，并额外使用3450亿词元来自通用数据集进行增强。我们在标准大语言模型基准测试、开放金融基准测试以及最能反映预期用途的内部基准测试套件上验证了BloombergGPT的性能。混合数据集训练使该模型在金融任务上显著优于现有模型，且未牺牲通用大语言模型基准测试的性能。此外，我们详细阐述了模型选择、训练过程及评估方法，并发布了附录C《训练编年史》，记录训练BloombergGPT的实践经验。