When building statistical models for Bayesian data analysis tasks, required and optional iterative adjustments and different modelling choices can give rise to numerous candidate models. In particular, checks and evaluations throughout the modelling process can motivate changes to an existing model or the consideration of alternative models to ultimately obtain models of sufficient quality for the problem at hand. Additionally, failing to consider alternative models can lead to overconfidence in the predictive or inferential ability of a chosen model. The search for suitable models requires modellers to work with multiple models without jeopardising the validity of their results. Multiverse analysis offers a framework for transparent creation of multiple models at once based on different sensible modelling choices, but the number of candidate models arising in the combination of iterations and possible modelling choices can become overwhelming in practice. Motivated by these challenges, this work proposes iterative filtering for multiverse analysis to support efficient and consistent assessment of multiple models and meaningful filtering towards fewer models of higher quality across different modelling contexts. Given that causal constraints have been taken into account, we show how multiverse analysis can be combined with recommendations from established Bayesian modelling workflows to identify promising candidate models by assessing predictive abilities and, if needed, tending to computational issues. We illustrate our suggested approach in different realistic modelling scenarios using real data examples.
翻译:在构建贝叶斯数据分析任务的统计模型时,必要的和可选的迭代调整以及不同的建模选择可能衍生出大量候选模型。特别是,整个建模过程中的检查与评估可能促使对现有模型进行修改或考虑替代模型,以最终获得满足问题要求的充分高质量模型。此外,忽视替代模型的考虑可能导致对所选择模型的预测或推断能力过度自信。寻找合适模型要求建模者在不影响结果有效性的前提下处理多个模型。多宇宙分析提供了一种框架,可基于不同合理的建模选择一次性透明地生成多个模型,但在实践中,由迭代与可能建模选择的组合所产生的大量候选模型可能变得难以应对。受这些挑战的启发,本文提出用于多宇宙分析的迭代筛选方法,以支持跨不同建模背景下对多个模型进行高效且一致的评估,并有意义地筛选出数量更少但质量更高的模型。在已考虑因果约束的前提下,我们展示了如何将多宇宙分析与既定贝叶斯建模工作流的建议相结合,通过评估预测能力(必要时处理计算问题)来识别有前景的候选模型。我们利用真实数据示例,在不同实际建模场景中说明了所建议的方法。