The domain adaptation of language models, including large language models (LLMs), has become increasingly important as the use of such models continues to expand. This study demonstrates the effectiveness of Composition to Augment Language Models (CALM) in adapting to the financial domain. CALM is a model to extend the capabilities of existing models by introducing cross-attention between two LLMs with different functions. In our experiments, we developed a CALM to enhance the financial performance of an LLM with strong response capabilities by leveraging a financial-specialized LLM. Notably, the CALM was trained using a financial dataset different from the one used to train the financial-specialized LLM, confirming CALM's ability to adapt to various datasets. The models were evaluated through quantitative Japanese financial benchmarks and qualitative response comparisons, demonstrating that CALM enables superior responses with higher scores than the original models and baselines. Additionally, comparative experiments on connection points revealed that connecting the middle layers of the models is most effective in facilitating adaptation to the financial domain. These findings confirm that CALM is a practical approach for adapting LLMs to the financial domain.

翻译：随着语言模型（包括大型语言模型，LLMs）的应用持续扩展，其领域适应性问题变得日益重要。本研究证明了组合增强语言模型（CALM）在适应金融领域方面的有效性。CALM是一种通过引入两个功能不同的LLM之间的交叉注意力来扩展现有模型能力的模型。在我们的实验中，我们开发了一个CALM，通过利用一个金融专用LLM来增强具有强大响应能力的LLM的金融性能。值得注意的是，CALM使用的训练数据集与训练金融专用LLM的数据集不同，这证实了CALM适应不同数据集的能力。通过定量的日本金融基准测试和定性的响应比较对模型进行评估，结果表明CALM能够实现优于原始模型和基线模型的响应，并获得更高的分数。此外，关于连接点的对比实验表明，连接模型的中间层对于促进金融领域适应最为有效。这些发现证实了CALM是将LLMs适应金融领域的一种实用方法。