Large language models (LLMs) have been incorporated into numerous industrial applications. Meanwhile, a vast array of API assets is scattered across various functions in the financial domain. An online financial question-answering system can leverage both LLMs and private APIs to provide timely financial analysis and information. The key is equipping the LLM model with function calling capability tailored to a financial scenario. However, a generic LLM requires customized financial APIs to call and struggles to adapt to the financial domain. Additionally, online user queries are diverse and contain out-of-distribution parameters compared with the required function input parameters, which makes it more difficult for a generic LLM to serve online users. In this paper, we propose a data-driven pipeline to enhance function calling in LLM for our online, deployed financial QA, comprising dataset construction, data augmentation, and model training. Specifically, we construct a dataset based on a previous study and update it periodically, incorporating queries and an augmentation method named AugFC. The addition of user query-related samples will \textit{exploit} our financial toolset in a data-driven manner, and AugFC explores the possible parameter values to enhance the diversity of our updated dataset. Then, we train an LLM with a two-step method, which enables the use of our financial functions. Extensive experiments on existing offline datasets, as well as the deployment of an online scenario, illustrate the superiority of our pipeline. The related pipeline has been adopted in the financial QA of YuanBao\footnote{https://yuanbao.tencent.com/chat/}, one of the largest chat platforms in China.

翻译：大语言模型已被广泛应用于众多工业应用中。与此同时，金融领域的大量API资产分散在各式各样的函数中。在线金融问答系统可同时利用大语言模型和私有API，提供及时的金融分析与信息。关键在于使大语言模型具备适配金融场景的函数调用能力。然而，通用大语言模型需要定制的金融API进行调用，且难以适应金融领域。此外，在线用户查询多样，且与所需函数输入参数相比存在分布外参数，这使得通用大语言模型更难服务在线用户。本文提出一种数据驱动的流水线，以增强大语言模型在已部署的在线金融问答中的函数调用能力，该流水线包括数据集构建、数据增强和模型训练。具体而言，我们基于先前研究构建数据集并定期更新，其中引入查询及名为AugFC的增强方法。以数据驱动方式添加用户查询相关样本，可充分利用金融工具集，AugFC则通过探索可能的参数值来增强更新后数据集的多样性。随后，我们采用两阶段方法训练大语言模型，使其能够调用金融函数。在现有离线数据集以及在线场景部署上的广泛实验表明，我们的流水线具有优越性。该流水线已应用于元宝的金融问答系统中（https://yuanbao.tencent.com/chat/），元宝是中国最大的聊天平台之一。