Multiverse analysis, a paradigm for statistical analysis that considers all combinations of reasonable analysis choices in parallel, promises to improve transparency and reproducibility. Although recent tools help analysts specify multiverse analyses, they remain difficult to use in practice. In this work, we identify debugging as a key barrier due to the latency from running analyses to detecting bugs and the scale of metadata processing needed to diagnose a bug. To address these challenges, we prototype a command-line interface tool, Multiverse Debugger, which helps diagnose bugs in the multiverse and propagate fixes. In a qualitative lab study (n=13), we use Multiverse Debugger as a probe to develop a model of debugging workflows and identify specific challenges, including difficulty in understanding the multiverse's composition. We conclude with design implications for future multiverse analysis authoring systems.

翻译：多元宇宙分析是一种并行考虑所有合理分析组合的统计分析方法，旨在提升透明度与可重复性。尽管现有工具能帮助分析师指定多元宇宙分析方案，但在实际应用中仍存在使用困难。本研究中，我们识别出调试过程是主要障碍，其根源在于从运行分析到检测错误的延迟，以及诊断错误所需的海量元数据处理规模。为应对这些挑战，我们开发了命令行界面工具Multiverse Debugger，用于诊断多元宇宙中的错误并传播修正方案。通过一项定性实验室研究（n=13），我们以Multiverse Debugger为探针，构建了调试工作流模型，并识别出具体挑战，包括理解多元宇宙构成时的困难。最后，我们为未来多元宇宙分析创作系统的设计提出启示。