As models are trained and deployed, developers need to be able to systematically debug errors that emerge in the machine learning pipeline. We present MDB, a debugging framework for interactively querying datasets and models. MDB integrates functional programming with relational algebra to build expressive queries over a database of datasets and model predictions. Queries are reusable and easily modified, enabling debuggers to rapidly iterate and refine queries to discover and characterize errors and model behaviors. We evaluate MDB on object detection, bias discovery, image classification, and data imputation tasks across self-driving videos, large language models, and medical records. Our experiments show that MDB enables up to 10x faster and 40\% shorter queries than other baselines. In a user study, we find developers can successfully construct complex queries that describe errors of machine learning models.

翻译：随着模型的训练与部署，开发者需要能够系统性地调试机器学习流程中出现的错误。我们提出MDB，一个用于交互式查询数据集与模型的调试框架。MDB将函数式编程与关系代数相结合，在数据集及模型预测的数据库上构建表达性查询。查询具有可复用且易修改的特性，使调试人员能够快速迭代和优化查询，从而发现并描述错误及模型行为。我们在自动驾驶视频、大型语言模型及医疗记录数据集上，针对目标检测、偏差发现、图像分类及数据插补任务对MDB进行评估。实验表明，与其他基线方法相比，MDB可实现高达10倍的查询速度提升，且查询语句缩短40%。用户研究显示，开发者能够成功构建描述机器学习模型错误的复杂查询。