Ensuring the long-term reproducibility of data analyses requires results stability tests to verify that analysis results remain within acceptable variation bounds despite inevitable software updates and hardware evolutions. This paper introduces a numerical variability approach for results stability tests, which determines acceptable variation bounds using random rounding of floating-point calculations. By applying the resulting stability test to \fmriprep, a widely-used neuroimaging tool, we show that the test is sensitive enough to detect subtle updates in image processing methods while remaining specific enough to accept numerical variations within a reference version of the application. This result contributes to enhancing the reliability and reproducibility of data analyses by providing a robust and flexible method for stability testing.
翻译:确保数据分析的长期可重复性需要结果稳定性测试,以验证分析结果在不可避免的软件更新和硬件演化下仍保持在可接受的变异范围内。本文提出了一种面向结果稳定性测试的数值变异性方法,该方法通过随机舍入浮点计算来界定可接受的变异范围。将所得到的稳定性测试应用于广泛使用的神经影像学工具\fmriprep,我们证明该测试具有足够的灵敏度以检测图像处理方法中的细微更新,同时保持足够的特异性以接受应用程序参考版本内的数值变异。这一结果通过提供一种稳健且灵活的稳定性测试方法,有助于提升数据分析的可靠性与可重复性。