As autonomous LLM agents evolve to perform complex analytical tasks that require integrating multiple datasets, the problem of discovering relevant data sources becomes a key bottleneck. Beyond the challenge posed by the sheer volume of available data sources, data-source selection is difficult because the semantics of data are nuanced and require considering many aspects of the data. To address this challenge, we introduce Metadata Reasoner, an agentic approach to metadata reasoning that identifies a small set of data sources that are both sufficient and minimal for a given analytical task. Metadata Reasoner leverages a table-search engine to retrieve candidate tables, and then autonomously consults various aspects of the available metadata to determine whether the candidates fit the requirements of the task. We demonstrate the effectiveness of Metadata Reasoner through a series of empirical studies. Evaluated on the real-world KramaBench datasets for data selection, our approach achieves an average F1 score of 88.63%, outperforming the strongest baseline by 10.73 percentage points. Furthermore, evaluations on a newly created synthetic benchmark based on the BIRD dataset reveal that Metadata Reasoner is highly robust against redundant and low-quality tables that are in the data lake. In this noisy environment, Metadata Reasoner maintains a 93.5% average F1 score for selecting the right datasets and demonstrates a 100% success rate in avoiding low-quality data. Finally, we extend Metadata Reasoner to support the interleaving of table selection with the execution of partial queries. Compared with a single pass of table selection, such interleaving improves table-selection F1 score on runtime-dependent tasks by 4.8 percentage points and end-to-end task accuracy by 5.3 points.

翻译：暂无翻译