False discovery rate (FDR) is commonly used for correction for multiple testing in neuroimaging studies. However, when using two-tailed tests, making directional inferences about the results can lead to vastly inflated error rate, even approaching 100\% in some cases. This happens because FDR only provides weak control over the error rate, meaning that the proportion of error is guaranteed only globally over all tests, not within subsets, such as among those in only one or another direction. Here we consider and evaluate different strategies for FDR control with two-tailed tests, using both synthetic and real imaging data. Approaches that separate the tests by direction of the hypothesis test, or by the direction of the resulting test statistic, more properly control the directional error rate and preserve FDR benefits, albeit with a doubled risk of errors under complete absence of signal. Strategies that combine tests in both directions, or that use simple two-tailed p-values, can lead to invalid directional conclusions, even if these tests remain globally valid. To enable valid thresholding for directional inference, we suggest that imaging software should allow the possibility that the user sets asymmetrical thresholds for the two sides of the statistical map. While FDR continues to be a valid, powerful procedure for multiple testing correction, care is needed when making directional inferences for two-tailed tests, or more broadly, when making any localized inference.
翻译:错误发现率(FDR)常用于神经影像学研究中多重检验的校正。然而,在使用双尾检验时,对结果进行方向性推断可能导致错误率大幅膨胀,在某些情况下甚至接近100%。这是因为FDR仅对错误率提供弱控制,即错误比例仅在整个检验集合全局层面得到保证,而非子集内(例如仅某一方向或另一方向的检验)。本文结合仿真和真实影像数据,评估了双尾检验中FDR控制的不同策略。按假设检验方向或检验统计量方向分离检验的方法能更合理地控制方向性错误率并保留FDR优势(尽管在完全无信号时错误风险加倍),而合并两个方向检验或使用简单双尾p值的策略可能导致无效的方向性结论(即便这些检验在全局层面仍然有效)。为支持有效的方向性推断阈值设定,我们建议影像软件允许用户对统计图两侧设置非对称阈值。尽管FDR仍是多重检验校正中有效且强有力的方法,但在对双尾检验进行方向性推断(或更广义的局部化推断)时仍需格外谨慎。