Permutation tests are widely recognized as robust alternatives to tests based on the normal theory. Random permutation tests have been frequently employed to assess the significance of variables in linear models. Despite their widespread use, existing random permutation tests lack finite-sample and assumption-free guarantees for controlling type I error in partial correlation tests. To address this standing challenge, we develop a conformal test through permutation-augmented regressions, which we refer to as PALMRT. PALMRT not only achieves power competitive with conventional methods but also provides reliable control of type I errors at no more than $2\alpha$ given any targeted level $\alpha$, for arbitrary fixed-designs and error distributions. We confirmed this through extensive simulations. Compared to the cyclic permutation test (CPT), which also offers theoretical guarantees, PALMRT does not significantly compromise power or set stringent requirements on the sample size, making it suitable for diverse biomedical applications. We further illustrate their differences in a long-Covid study where PALMRT validated key findings previously identified using the t-test, while CPT suffered from a drastic loss of power. We endorse PALMRT as a robust and practical hypothesis test in scientific research for its superior error control, power preservation, and simplicity.
翻译:置换检验被广泛认为是基于正态理论检验的稳健替代方法。随机置换检验常被用于评估线性模型中变量的显著性。尽管应用广泛,现有随机置换检验在偏相关检验中缺乏控制第一类错误率的有限样本和无假设保证。为应对这一长期挑战,我们通过置换增广回归开发了一种共形检验,称之为PALMRT。PALMRT不仅实现了与传统方法相当的功效,还能在任意固定设计和误差分布下,针对任意目标水平α,将第一类错误率可靠控制在不超过2α。我们通过大量仿真验证了这一点。与同样提供理论保证的循环置换检验(CPT)相比,PALMRT不会显著牺牲功效或对样本量设置苛刻要求,使其适用于多样化的生物医学应用。我们进一步在长期新冠研究中展示了二者的差异:PALMRT验证了先前基于t检验识别的关键发现,而CPT则出现严重的功效损失。鉴于其优越的错误控制能力、功效保持性及简便性,我们推荐PALMRT作为科学研究中稳健实用的假设检验方法。