Advances in large language models (LLMs) have created new opportunities in data science, but their deployment is often limited by the challenge of finding relevant data in large data lakes. Existing methods struggle with this: both single- and multi-agent systems are quickly overwhelmed by large, heterogeneous files, and master-slave multi-agent systems rely on a rigid central controller that requires precise knowledge of each sub-agent's capabilities, which is not possible in large-scale settings where the main agent lacks full observability over sub-agents' knowledge and competencies. We propose a novel multi-agent paradigm inspired by the blackboard architecture for traditional AI models. In our framework, a central agent posts requests to a shared blackboard, and autonomous subordinate agents - either responsible for a partition of the data lake or retrieval from the web - volunteer to respond based on their capabilities. This design improves scalability and flexibility by removing the need for a central coordinator to know each agent's expertise or internal knowledge. We evaluate the approach on three benchmarks that require data discovery: KramaBench and modified versions of DSBench and DA-Code. Results show that the blackboard architecture substantially outperforms strong baselines, achieving 13%-57% relative improvements in end-to-end success and up to a 9% relative gain in data discovery F1 over the best baseline.

翻译：大型语言模型（LLM）的进展为数据科学创造了新的机遇，但其部署往往受限于在大型数据湖中寻找相关数据的挑战。现有方法难以应对这一挑战：无论是单智能体还是多智能体系统，在面对大规模异构文件时都迅速不堪重负；而主从式多智能体系统依赖于僵化的中央控制器，该控制器需要精确了解每个子智能体的能力——这在主智能体无法完全观测子智能体知识与能力的大规模场景中是无法实现的。我们提出了一种受传统AI模型黑板架构启发的新型多智能体范式。在我们的框架中，中央智能体将请求发布到共享黑板，而负责数据湖分区或网络检索的自主从属智能体则根据自身能力主动响应。该设计通过消除中央协调器了解每个智能体专业知识或内部知识的需要，提高了可扩展性和灵活性。我们在三个需要数据发现的基准测试上评估了该方法：KramaBench以及修改版的DSBench和DA-Code。结果表明，黑板架构显著优于强基线模型，在端到端成功率上实现了13%-57%的相对提升，在数据发现F1分数上相比最佳基线获得了最高9%的相对增益。