With the rapid progress of large language models (LLMs), financial information retrieval has become a critical industrial application. Extracting task-relevant information from lengthy financial filings is essential for both operational and analytical decision-making. We present PRISM, a training-free framework that integrates refined system prompting, in-context learning (ICL), and lightweight multi-agent coordination for document and chunk ranking tasks. Our primary contribution is a systematic empirical study of when each component provides value: prompt engineering delivers consistent performance with minimal overhead, ICL enhances reasoning for complex queries when applied selectively, and multi-agent systems show potential primarily with larger models and careful architectural design. Extensive ablation studies across FinAgentBench, FiQA-2018, and FinanceBench reveal that simpler configurations often outperform complex multi-agent pipelines, providing practical guidance for practitioners. Our best configuration achieves an NDCG@5 of 0.71818 on FinAgentBench, ranking third while being the only training-free approach in the top three. We provide comprehensive feasibility analyses covering latency, token usage, and cost trade-offs to support deployment decisions. The source code is released at https://bit.ly/prism-ailens.
翻译:随着大语言模型(LLMs)的快速发展,金融信息检索已成为一项关键的工业应用。从冗长的金融文件中提取任务相关信息,对于运营和分析决策均至关重要。我们提出PRISM,一种无需训练的新框架,它集成了精炼的系统提示、上下文学习(ICL)以及轻量级多智能体协调,用于文档和段落排序任务。我们的主要贡献在于对各组件何时产生价值进行了系统的实证研究:提示工程能以最小开销带来稳定性能;选择性应用ICL可增强对复杂查询的推理能力;而多智能体系统主要在与更大模型及精心设计的架构结合时才展现潜力。在FinAgentBench、FiQA-2018和FinanceBench上的大量消融研究表明,简单的配置通常优于复杂的多智能体流水线,为从业者提供了实用指导。我们的最佳配置在FinAgentBench上达到0.71818的NDCG@5,排名第三,且是前三名中唯一无需训练的方法。我们提供了涵盖延迟、令牌用量及成本权衡等全面的可行性分析,以支持部署决策。源代码发布于https://bit.ly/prism-ailens。