We investigate the reasoning capabilities of large language models (LLMs) for automatically generating data-cleaning workflows. To evaluate LLMs' ability to complete data-cleaning tasks, we implemented a pipeline for LLM-based Auto Data Cleaning Workflow (AutoDCWorkflow), prompting LLMs on data cleaning operations to repair three types of data quality issues: duplicates, missing values, and inconsistent data formats. Given a dirty table and a purpose (expressed as a query), this pipeline generates a minimal, clean table sufficient to address the purpose and the data cleaning workflow used to produce the table. The planning process involves three main LLM-driven components: (1) Select Target Columns: Identifies a set of target columns related to the purpose. (2) Inspect Column Quality: Assesses the data quality for each target column and generates a Data Quality Report as operation objectives. (3) Generate Operation & Arguments: Predicts the next operation and arguments based on the data quality report results. Additionally, we propose a data cleaning benchmark to evaluate the capability of LLM agents to automatically generate workflows that address data cleaning purposes of varying difficulty levels. The benchmark comprises the annotated datasets as a collection of purpose, raw table, clean table, data cleaning workflow, and answer set. In our experiments, we evaluated three LLMs that auto-generate purpose-driven data cleaning workflows. The results indicate that LLMs perform well in planning and generating data-cleaning workflows without the need for fine-tuning.

翻译：本文研究了大语言模型（LLM）在自动生成数据清洗工作流方面的推理能力。为了评估LLM完成数据清洗任务的能力，我们实现了一个基于LLM的自动数据清洗工作流（AutoDCWorkflow）管道，通过提示LLM执行数据清洗操作来修复三类数据质量问题：重复值、缺失值和不一致的数据格式。给定一个脏数据表和一个目标（以查询形式表达），该管道会生成一个足以满足该目标的最小化干净数据表，以及用于生成该表的数据清洗工作流。其规划过程包含三个主要的LLM驱动组件：（1）选择目标列：识别与目标相关的一组目标列。（2）检查列质量：评估每个目标列的数据质量，并生成作为操作目标的数据质量报告。（3）生成操作与参数：根据数据质量报告结果预测下一个操作及其参数。此外，我们提出了一个数据清洗基准测试，用于评估LLM智能体自动生成应对不同难度级别数据清洗目标的工作流的能力。该基准测试包含带标注的数据集，集成了目标、原始表、干净表、数据清洗工作流和答案集合。在我们的实验中，我们评估了三种能够自动生成目标驱动数据清洗工作流的LLM。结果表明，LLM在无需微调的情况下，在规划和生成数据清洗工作流方面表现良好。