Large language models (LLMs) have exhibited an array of reasoning capabilities but face challenges like error propagation and hallucination, particularly in specialised areas like finance, where data is heterogeneous, and precision is paramount. We explore the potential of language model augmentation with external tools to mitigate these limitations and offload certain reasoning steps to external tools that are more suited for the task, instead of solely depending on the LLM's inherent abilities. More concretely, using financial domain question-answering datasets, we apply supervised fine-tuning on a LLaMA-2 13B Chat model to act both as a 'task router' and 'task solver'. The 'task router' dynamically directs a question to either be answered internally by the LLM or externally via the right tool from the tool set. Our tool-equipped SFT model, Raven, demonstrates an improvement of 35.2% and 5.06% over the base model and SFT-only baselines, respectively, and is highly competitive with strong GPT-3.5 results. To the best of our knowledge, our work is the first that investigates tool augmentation of language models for the finance domain.

翻译：大型语言模型（LLMs）展现了一系列推理能力，但在金融等专业领域仍面临错误传播和幻觉等问题——这些领域数据异构且对精度要求极高。我们探索通过外部工具增强语言模型的方法来缓解这些局限性，将部分推理步骤交由更擅长的外部工具执行，而非完全依赖LLM的内在能力。具体而言，我们利用金融领域问答数据集，对LLaMA-2 13B聊天模型进行监督微调，使其兼具"任务路由"与"任务求解"双重功能。"任务路由"可动态引导问题由LLM内部解答，或通过工具集中的合适工具进行外部解决。基于此方法构建的Raven模型，相较基础模型和纯微调基线分别提升35.2%和5.06%，与强基准GPT-3.5结果相比极具竞争力。据我们所知，本研究首次探索了针对金融领域的语言模型工具增强方法。