Audio Large Language Models (AudioLLMs) have received widespread attention and have significantly improved performance on audio tasks such as conversation, audio understanding, and automatic speech recognition (ASR). Despite these advancements, there is an absence of a benchmark for assessing AudioLLMs in financial scenarios, where audio data, such as earnings conference calls and CEO speeches, are crucial resources for financial analysis and investment decisions. In this paper, we introduce \textsc{FinAudio}, the first benchmark designed to evaluate the capacity of AudioLLMs in the financial domain. We first define three tasks based on the unique characteristics of the financial domain: 1) ASR for short financial audio, 2) ASR for long financial audio, and 3) summarization of long financial audio. Then, we curate two short and two long audio datasets, respectively, and develop a novel dataset for financial audio summarization, comprising the \textsc{FinAudio} benchmark. Then, we evaluate seven prevalent AudioLLMs on \textsc{FinAudio}. Our evaluation reveals the limitations of existing AudioLLMs in the financial domain and offers insights for improving AudioLLMs. All datasets and codes will be released.

翻译：音频大语言模型（AudioLLMs）已获得广泛关注，并在对话、音频理解、自动语音识别（ASR）等音频任务上取得了显著性能提升。尽管取得了这些进展，目前仍缺乏用于评估金融场景下AudioLLMs性能的基准测试。在金融分析及投资决策中，诸如财报电话会议和CEO演讲等音频数据是至关重要的资源。本文提出了首个用于评估金融领域AudioLLMs能力的基准测试——\textsc{FinAudio}。我们首先基于金融领域的独特性定义了三大任务：1）短金融音频的ASR，2）长金融音频的ASR，以及3）长金融音频的摘要生成。随后，我们分别构建了两个短音频数据集和两个长音频数据集，并开发了一个新颖的金融音频摘要数据集，共同构成了\textsc{FinAudio}基准测试。接着，我们在\textsc{FinAudio}上评估了七种主流的AudioLLMs。评估结果揭示了现有AudioLLMs在金融领域的局限性，并为改进AudioLLMs提供了洞见。所有数据集与代码均将开源发布。