Financial large language models (FinLLMs) have been applied to various tasks in business, finance, accounting, and auditing. Complex financial regulations and standards are critical to financial services, which LLMs must comply with. However, FinLLMs' performance in understanding and interpreting financial regulations has rarely been studied. Therefore, we organize the Regulations Challenge, a shared task at COLING 2025. It encourages the academic community to explore the strengths and limitations of popular LLMs. We create 9 novel tasks and corresponding question sets. In this paper, we provide an overview of these tasks and summarize participants' approaches and results. We aim to raise awareness of FinLLMs' professional capability in financial regulations.
翻译:金融大语言模型(FinLLMs)已被应用于商业、金融、会计和审计领域的各种任务。复杂的金融法规和标准对金融服务至关重要,大语言模型必须遵守这些规定。然而,FinLLMs在理解和解释金融法规方面的性能却鲜有研究。为此,我们在COLING 2025上组织了"法规挑战赛"这一共享任务,旨在鼓励学术界探索主流大语言模型的优势与局限。我们创建了9项新颖的任务及相应的问题集。本文概述了这些任务,并总结了参赛者的方法及结果,以期提升业界对FinLLMs在金融法规领域专业能力的关注。