In contemporary problems involving genetic or neuroimaging data, thousands of hypotheses need to be tested. Due to their high power, and finite sample guarantees on type-1 error under weak assumptions, Monte-Carlo permutation tests are often considered as gold standard for these settings. However, the enormous computational effort required for (thousands of) permutation tests is a major burden. Recently, Fischer and Ramdas (2024) constructed a permutation test for a single hypothesis in which the permutations are drawn sequentially one-by-one and the testing process can be stopped at any point without inflating the type I error. They showed that the number of permutations can be substantially reduced (under null and alternative) while the power remains similar. We show how their approach can be modified to make it suitable for a broad class of multiple testing procedures. In particular, we discuss its use with the Benjamini-Hochberg procedure and illustrate the application on a large dataset.

翻译：在涉及遗传学或神经影像数据的当代问题中，常需检验成千上万个假设。蒙特卡洛置换检验因其高统计功效，以及在弱假设下对第一类错误的有限样本保证，常被视为这些场景下的黄金标准。然而，执行（数千次）置换检验所需的巨大计算量是一个主要负担。最近，Fischer和Ramdas（2024）构建了一种针对单一假设的置换检验，其中置换被逐个顺序抽取，且检验过程可在任意时刻停止而不会增大第一类错误。他们证明，在保持统计功效相近的同时，（在原假设和备择假设下）置换次数可被显著减少。我们展示了如何修改他们的方法，使其适用于一大类多重检验程序。具体而言，我们讨论了其与Benjamini-Hochberg程序的结合使用，并在一大型数据集上展示了应用实例。