Ensuring the long-term reproducibility of data analyses requires results stability tests to verify that analysis results remain within acceptable variation bounds despite inevitable software updates and hardware evolutions. This paper introduces a numerical variability approach for results stability tests, which determines acceptable variation bounds using random rounding of floating-point calculations. By applying the resulting stability test to \fmriprep, a widely-used neuroimaging tool, we show that the test is sensitive enough to detect subtle updates in image processing methods while remaining specific enough to accept numerical variations within a reference version of the application. This result contributes to enhancing the reliability and reproducibility of data analyses by providing a robust and flexible method for stability testing.
翻译:确保数据分析的长期可重复性需要结果稳定性检验,以验证分析结果在不可避免的软件更新和硬件演进背景下仍处于可接受的变异范围内。本文提出了一种基于数值变异性的结果稳定性检验方法,该方法通过采用浮点运算的随机舍入来确定可接受的变异范围。将该稳定性检验方法应用于广泛使用的神经影像工具\fmriprep后,我们证明该检验方法具有足够高的灵敏度以检测图像处理方法中的细微更新,同时保持足够的特异性以接受同一应用程序参考版本内的数值变异。这一结果为增强数据分析的可靠性和可重复性提供了稳健且灵活的稳定性检验方法。