Machine learning's influence is expanding rapidly, now integral to decision-making processes from corporate strategy to the advancements in Industry 4.0. The efficacy of Artificial Intelligence broadly hinges on the caliber of data used during its training phase; optimal performance is tied to exceptional data quality. Data cleaning tools, particularly those that exploit functional dependencies within ontological frameworks or context models, are instrumental in augmenting data quality. Nevertheless, crafting these context models is a demanding task, both in terms of resources and expertise, often necessitating specialized knowledge from domain experts. In light of these challenges, this paper introduces an innovative approach, called LLMClean, for the automated generation of context models, utilizing Large Language Models to analyze and understand various datasets. LLMClean encompasses a sequence of actions, starting with categorizing the dataset, extracting or mapping relevant models, and ultimately synthesizing the context model. To demonstrate its potential, we have developed and tested a prototype that applies our approach to three distinct datasets from the Internet of Things, healthcare, and Industry 4.0 sectors. The results of our evaluation indicate that our automated approach can achieve data cleaning efficacy comparable with that of context models crafted by human experts.
翻译:机器学习的影响力正迅速扩展,如今已深度融入从企业战略到工业4.0发展的决策过程。人工智能的整体效能高度依赖于训练阶段的数据质量——最优性能与卓越的数据质量紧密相连。数据清洗工具,尤其是利用本体框架或上下文模型中的函数依赖关系的工具,在提升数据质量方面发挥着关键作用。然而,构建这些上下文模型在资源和专业知识层面均要求极高,通常需要领域专家的专门知识。针对这些挑战,本文提出了一种创新方法LLMClean,利用大型语言模型自动生成上下文模型,以分析和理解各类数据集。LLMClean包含一系列步骤:首先对数据集进行分类,然后提取或映射相关模型,最终综合生成上下文模型。为验证其潜力,我们开发并测试了一个原型系统,将其应用于来自物联网、医疗保健和工业4.0三个领域的独立数据集。评估结果表明,我们的自动化方法能够达到与人类专家构建的上下文模型相媲美的数据清洗效果。