Developing machine learning (ML) models requires a deep understanding of real-world problems, which are inherently multi-objective. In this paper, we present VirnyFlow, the first design space for responsible model development, designed to assist data scientists in building ML pipelines that are tailored to the specific context of their problem. Unlike conventional AutoML frameworks, VirnyFlow enables users to define customized optimization criteria, perform comprehensive experimentation across pipeline stages, and iteratively refine models in alignment with real-world constraints. Our system integrates evaluation protocol definition, multi-objective Bayesian optimization, cost-aware multi-armed bandits, query optimization, and distributed parallelism into a unified architecture. We show that VirnyFlow significantly outperforms state-of-the-art AutoML systems in both optimization quality and scalability across five real-world benchmarks, offering a flexible, efficient, and responsible alternative to black-box automation in ML development.

翻译：开发机器学习模型需要深入理解现实世界问题，这些问题本质上是多目标的。本文提出VirnyFlow——首个面向负责任模型开发的设计空间，旨在帮助数据科学家构建针对特定问题情境量身定制的机器学习流程。与传统AutoML框架不同，VirnyFlow允许用户定义定制化优化准则，在流程各阶段进行全面实验，并根据现实约束迭代优化模型。本系统将评估协议定义、多目标贝叶斯优化、成本感知多臂老虎机、查询优化和分布式并行技术集成于统一架构。通过在五个现实基准测试中的实验证明，VirnyFlow在优化质量和可扩展性方面均显著优于当前最先进的AutoML系统，为机器学习开发提供了灵活、高效且负责任的替代方案，以取代黑盒自动化范式。