Nowadays, machine learning (ML) plays a vital role in many aspects of our daily life. In essence, building well-performing ML applications requires the provision of high-quality data throughout the entire life-cycle of such applications. Nevertheless, most of the real-world tabular data suffer from different types of discrepancies, such as missing values, outliers, duplicates, pattern violation, and inconsistencies. Such discrepancies typically emerge while collecting, transferring, storing, and/or integrating the data. To deal with these discrepancies, numerous data cleaning methods have been introduced. However, the majority of such methods broadly overlook the requirements imposed by downstream ML models. As a result, the potential of utilizing these data cleaning methods in ML pipelines is predominantly unrevealed. In this work, we introduce a comprehensive benchmark, called REIN1, to thoroughly investigate the impact of data cleaning methods on various ML models. Through the benchmark, we provide answers to important research questions, e.g., where and whether data cleaning is a necessary step in ML pipelines. To this end, the benchmark examines 38 simple and advanced error detection and repair methods. To evaluate these methods, we utilized a wide collection of ML models trained on 14 publicly-available datasets covering different domains and encompassing realistic as well as synthetic error profiles.

翻译：如今，机器学习在日常生活众多方面扮演着关键角色。本质上，构建性能优异的机器学习应用需在整个应用生命周期中提供高质量数据。然而，多数真实世界的表格数据都遭受不同类型的数据瑕疵，如缺失值、异常值、重复值、模式违规及不一致性。这些瑕疵通常在数据收集、传输、存储和/或整合过程中产生。为应对这些瑕疵，学界已提出诸多数据清洗方法。但大多数此类方法普遍忽略了下游机器学习模型的需求，导致其在机器学习管道中的应用潜力尚未充分揭示。本研究提出名为REIN的综合基准框架，系统探讨数据清洗方法对各类机器学习模型的影响。通过该基准，我们回答了关键研究问题（例如：机器学习管道中数据清洗的必要性及其适用场景）。为此，基准评估了38种简单与高级的错误检测与修复方法。为验证这些方法，我们使用了涵盖不同领域的14个公开数据集训练的广泛机器学习模型，这些数据集包含真实及合成错误特征。