Understanding dataset semantics is crucial for effective search, discovery, and integration pipelines. To this end, column-type annotation (CTA) methods associate columns in tabular datasets with semantic types that accurately describe their contents, using pretrained deep learning models or Large Language Models (LLMs). However, existing approaches have important limitations. First, they require users to specify a closed set of semantic types either during training or inference, which hinders their application to domain-specific datasets where predefined type vocabularies often lack adequate coverage and specificity. They also assume that each column is associated with a single semantic type, an assumption that does not hold for many real-world datasets. Lastly, while proprietary LLMs have shown effectiveness for CTA, they incur high monetary costs and produce inconsistent outputs across similar columns, leading to type redundancy that degrades downstream applications. To address these challenges, we introduce StraTyper, a cost-effective method for column type discovery (CTD) and multi-type annotation (CMTA) in dataset collections. StraTyper eliminates the need for predefined semantic labels by employing LLMs to discover types tailored to the target dataset collection. Through strategic column clustering, controlled type generation, and iterative cascading discovery, StraTyper balances type precision with annotation coverage while minimizing LLM costs. Our experimental evaluation, both manual and LLM-assisted, on real-world dataset collections demonstrates that StraTyper discovers accurate types for both numeric and textual data, achieves substantial cost savings over commercial LLMs, and effectively handles multi-typed columns. We further show that StraTyper's annotations improve downstream tasks, including join discovery and schema matching, outperforming LLM-only baselines.
翻译:暂无翻译