Current automated machine learning (ML) tools are model-centric, focusing on model selection and parameter optimization. However, the majority of the time in data analysis is devoted to data cleaning and wrangling, for which limited tools are available. Here we present DataAssist, an automated data preparation and cleaning platform that enhances dataset quality using ML-informed methods. We show that DataAssist provides a pipeline for exploratory data analysis and data cleaning, including generating visualization for user-selected variables, unifying data annotation, suggesting anomaly removal, and preprocessing data. The exported dataset can be readily integrated with other autoML tools or user-specified model for downstream analysis. Our data-centric tool is applicable to a variety of fields, including economics, business, and forecasting applications saving over 50\% time of the time spent on data cleansing and preparation.
翻译:当前自动化机器学习工具以模型为中心,主要侧重于模型选择与参数优化。然而,数据分析中的大部分时间实际耗费在数据清洗与整理环节,而相关工具却十分有限。本文提出DataAssist——一个基于机器学习方法自动化数据准备与清洗平台,能够有效提升数据集质量。我们展示DataAssist提供了一套包含探索性数据分析与数据清洗的完整流程,具体功能包括:为用户选定变量生成可视化图表、统一数据标注格式、智能识别异常值并建议移除、以及数据预处理。导出的数据集可无缝对接其他自动机器学习工具或用户自定义模型进行下游分析。这种以数据为中心的工具可广泛应用于经济学、商业及预测等领域,能够节省超过50%的数据清洗与准备时间。