Multiverse analysis, a paradigm for statistical analysis that considers all combinations of reasonable analysis choices in parallel, promises to improve transparency and reproducibility. Although recent tools help analysts specify multiverse analyses, they remain difficult to use in practice. In this work, we identify debugging as a key barrier due to the latency from running analyses to detecting bugs and the scale of metadata processing needed to diagnose a bug. To address these challenges, we prototype a command-line interface tool, Multiverse Debugger, which helps diagnose bugs in the multiverse and propagate fixes. In a qualitative lab study (n=13), we use Multiverse Debugger as a probe to develop a model of debugging workflows and identify specific challenges, including difficulty in understanding the multiverse's composition. We conclude with design implications for future multiverse analysis authoring systems.

翻译：多宇宙分析是一种考虑所有合理分析选择的并行组合的统计分析方法，旨在提高透明度和可重复性。尽管现有工具能帮助分析师指定多宇宙分析，但在实际应用中仍难以使用。在本工作中，我们识别出调试是关键障碍，其原因在于从运行分析到检测到错误的延迟，以及诊断错误所需的大规模元数据处理。为应对这些挑战，我们原型实现了一个命令行界面工具——多宇宙调试器（Multiverse Debugger），该工具帮助诊断多宇宙中的错误并传播修复方案。通过一项定性实验室研究（n=13），我们以多宇宙调试器为探针，构建了调试工作流模型，并识别了具体挑战，包括难以理解多宇宙的构成。最后，我们为未来多宇宙分析撰写系统提出了设计启示。