Data drifts pose a critical challenge in the lifecycle of machine learning (ML) models, affecting their performance and reliability. In response to this challenge, we present a microbenchmark study, called D3Bench, which evaluates the efficacy of open-source drift detection tools. D3Bench examines the capabilities of Evidently AI, NannyML, and Alibi-Detect, leveraging real-world data from two smart building use cases.We prioritize assessing the functional suitability of these tools to identify and analyze data drifts. Furthermore, we consider a comprehensive set of non-functional criteria, such as the integrability with ML pipelines, the adaptability to diverse data types, user-friendliness, computational efficiency, and resource demands. Our findings reveal that Evidently AI stands out for its general data drift detection, whereas NannyML excels at pinpointing the precise timing of shifts and evaluating their consequent effects on predictive accuracy.

翻译：数据漂移是机器学习模型生命周期中面临的关键挑战，严重影响模型的性能和可靠性。针对这一问题，我们提出了一项名为D3Bench的微基准研究，用于评估开源漂移检测工具的有效性。该研究基于两个智能建筑用例的真实数据，对Evidently AI、NannyML和Alibi-Detect三款工具的功能进行了系统评估。我们优先评估这些工具在识别和分析数据漂移方面的功能适用性，同时综合考虑了与机器学习管道的集成能力、对多样化数据类型的适应性、用户友好性、计算效率及资源需求等非功能性指标。研究结果表明：Evidently AI在通用数据漂移检测方面表现突出，而NannyML则擅长精确定位漂移发生的时间节点，并评估其对预测准确性的后续影响。