In contemporary problems involving genetic or neuroimaging data, thousands of hypotheses need to be tested. Due to their high power, and finite sample guarantees on type-I error under weak assumptions, Monte-Carlo permutation tests are often considered as gold standard for these settings. However, the enormous computational effort required for (thousands of) permutation tests is a major burden. Recently, Fischer and Ramdas (2024) constructed a permutation test for a single hypothesis in which the permutations are drawn sequentially one-by-one and the testing process can be stopped at any point without inflating the type-I error. They showed that the number of permutations can be substantially reduced (under null and alternative) while the power remains similar. We show how their approach can be modified to make it suitable for a broad class of multiple testing procedures and particularly discuss its use with the Benjamini-Hochberg procedure. The resulting method provides valid error rate control and outperforms all existing approaches significantly in terms of power and/or required computational time. We provide fast implementations and illustrate its application on large datasets, both synthetic and real.

翻译：在涉及遗传或神经影像数据的当代问题中，常需检验数千个假设。蒙特卡洛置换检验因其高功效，以及在弱假设下对第一类错误的有限样本保证，常被视为这些场景下的黄金标准。然而，执行（数千次）置换检验所需的巨大计算量是一个主要负担。最近，Fischer与Ramdas（2024）构建了一种针对单一假设的置换检验，其中置换被逐个顺序抽取，且检验过程可在任意时刻停止而不会增大第一类错误。他们证明，置换次数可大幅减少（在原假设和备择假设下均如此），同时功效保持相近。我们展示了如何修改其方法，使其适用于一大类多重检验程序，并特别讨论了其与Benjamini-Hochberg程序的结合使用。所得方法能提供有效的错误率控制，并在功效和/或所需计算时间方面显著优于所有现有方法。我们提供了快速实现，并在合成与真实的大型数据集上说明了其应用。