Monte Carlo permutation tests are a cornerstone of valid, model-free statistical inference. A widely held practical intuition is that increasing the number of sampled permutations improves test performance, in particular that statistical power tends to increase with the Monte Carlo budget. In this paper, we show that these intuitions are false in general. Leveraging the saw-toothed structure of power arising from distributional discreteness, we provide a simple structural explanation for why power can decrease as the number of sampled permutations increases, and we prove that such decreases occur infinitely often as the Monte Carlo budget grows.
翻译:蒙特卡洛置换检验是有效、无模型统计推断的基石。一种普遍存在的实践直觉认为,增加抽样排列的次数可提升检验性能,尤其是统计功效往往会随蒙特卡洛预算的增加而增强。本文证明这些直觉在一般情况下是错误的。利用由分布离散性导致的功效锯齿形结构,我们为当抽样排列次数增加时功效为何会下降提供了一个简单的结构性解释,并证明了随着蒙特卡洛预算的增长,这种下降会无限频繁地发生。