Natural language processing (NLP) has recently gained relevance within financial institutions by providing highly valuable insights into companies and markets' financial documents. However, the landscape of the financial domain presents extra challenges for NLP, due to the complexity of the texts and the use of specific terminology. Generalist language models tend to fall short in tasks specifically tailored for finance, even when using large language models (LLMs) with great natural language understanding and generative capabilities. This paper presents a study on LLM adaptation methods targeted at the financial domain and with high emphasis on financial sentiment analysis. To this purpose, two foundation models with less than 1.5B parameters have been adapted using a wide range of strategies. We show that through careful fine-tuning on both financial documents and instructions, these foundation models can be adapted to the target domain. Moreover, we observe that small LLMs have comparable performance to larger scale models, while being more efficient in terms of parameters and data. In addition to the models, we show how to generate artificial instructions through LLMs to augment the number of samples of the instruction dataset.

翻译：自然语言处理（NLP）近期通过从公司及市场的金融文档中提取高价值见解，已在金融机构中展现出相关性。然而，由于文本复杂性及特定术语的使用，金融领域的格局为NLP带来了额外挑战。通用型语言模型在专为金融任务定制的场景中往往表现不足，即使是具备强大自然语言理解与生成能力的大语言模型（LLMs）也不例外。本文针对金融领域，重点聚焦金融情感分析，对LLM适配方法展开研究。为此，我们采用多种策略适配了两个参数规模低于15亿的基础模型。研究表明，通过对金融文档与指令进行精细微调，这些基础模型能够成功适配目标领域。此外，我们观察到小规模LLM在参数与数据效率上更优的同时，其性能可与大规模模型相媲美。除模型外，我们还展示了如何通过LLM生成人工指令以扩充指令数据集的样本数量。