Hallucinations are inevitable in downstream tasks using large language models (LLMs). While addressing hallucinations becomes a substantial challenge for LLM-based ontology matching (OM) systems, we introduce a new benchmark dataset called OAEI-LLM-T. The dataset evolves from the TBox (i.e. schema-matching) datasets in the Ontology Alignment Evaluation Initiative (OAEI), capturing hallucinations of different LLMs performing OM tasks. These OM-specific hallucinations are carefully classified into two primary categories and six sub-categories. We showcase the usefulness of the dataset in constructing the LLM leaderboard and fine-tuning foundational LLMs for LLM-based OM systems.

翻译：在使用大语言模型的下游任务中，幻觉是不可避免的。虽然解决幻觉问题已成为基于大语言模型的本体匹配系统面临的一项重大挑战，但我们引入了一个名为OAEI-LLM-T的新基准数据集。该数据集源自本体对齐评估倡议中的TBox数据集，旨在捕捉不同大语言模型在执行本体匹配任务时产生的幻觉。这些特定于本体匹配的幻觉被细致地划分为两个主要类别和六个子类别。我们展示了该数据集在构建大语言模型排行榜以及为基于大语言模型的本体匹配系统微调基础大语言模型方面的实用性。