Software engineers must make decisions that trade off competing goals (faster vs. cheaper, secure vs. usable, accurate vs. interpretable, etc.). Despite MSR's proven techniques for exploring such goals, researchers still struggle with these trade-offs. Similarly, industrial practitioners deliver sub-optimal products since they lack the tools needed to explore these trade-offs. To address this, MOOT (http://tiny.cc/moot) is a repository of many SE multi-objective optimization tasks. MOOT's 120+ tasks cover software configuration, cloud tuning, project health, process modeling, hyperparameter optimization, and more. Sample scripts for reading MOOT and generating baseline results are available -- just clone the repository and run the sample rqx.sh files (from tiny.cc/moot0). To the best of our knowledge, MOOT is the largest and most varied collection of real multi-objective optimization tasks in SE. We note that MOOT's novelty is infrastructural, not algorithmic-we contribute curated data and research enablement, not new optimization methods. MOOT enables harder and more credible research. MOOT lets us replace studies on toy problems (or just half a dozen hand-picked examples) with case studies on 120+ examples. Such studies could focus on stability, sample efficiency, failure modes, cross-domain generality, or many other questions (see list in this document).
翻译:软件工程师在决策时必须权衡相互竞争的目标(速度与成本、安全性与可用性、准确性与可解释性等)。尽管软件工程领域已有成熟的探索此类目标的技术,研究人员仍难以有效处理这些权衡问题。同样,工业界从业者由于缺乏探索这些权衡的必要工具,往往交付次优产品。为解决这一问题,MOOT(http://tiny.cc/moot)构建了一个包含大量软件工程多目标优化任务的存储库。MOOT收录的120余项任务涵盖软件配置、云环境调优、项目健康度评估、流程建模、超参数优化等多个领域。我们提供了读取MOOT数据及生成基准结果的示例脚本——只需克隆该存储库并运行示例rqx.sh文件(获取地址:tiny.cc/moot0)。据我们所知,MOOT是当前软件工程领域规模最大、类型最丰富的真实多目标优化任务集合。需要说明的是,MOOT的创新性体现在基础设施层面而非算法层面——我们贡献的是经过系统整理的数据与研究支持工具,而非新的优化方法。MOOT能够支持更具挑战性和可信度的研究,使我们能够用120多个实际案例研究替代以往基于简单模拟问题(或仅少数手工挑选示例)的研究范式。此类研究可聚焦于算法稳定性、采样效率、失效模式、跨领域泛化能力等诸多方向(具体研究方向清单详见本文档)。