Many testing problems are readily amenable to randomised tests such as those employing data splitting. However despite their usefulness in principle, randomised tests have obvious drawbacks. Firstly, two analyses of the same dataset may lead to different results. Secondly, the test typically loses power because it does not fully utilise the entire sample. As a remedy to these drawbacks, we study how to combine the test statistics or p-values resulting from multiple random realisations such as through random data splits. We develop rank-transformed subsampling as a general method for delivering large sample inference about the combined statistic or p-value under mild assumptions. We apply our methodology to a wide range of problems, including testing unimodality in high-dimensional data, testing goodness-of-fit of parametric quantile regression models, testing no direct effect in a sequentially randomised trial and calibrating cross-fit double machine learning confidence intervals. In contrast to existing p-value aggregation schemes that can be highly conservative, our method enjoys type-I error control that asymptotically approaches the nominal level. Moreover, compared to using the ordinary subsampling, we show that our rank transform can remove the first-order bias in approximating the null under alternatives and greatly improve power.

翻译：许多检验问题易于采用随机化检验方法，如利用数据分割的检验。然而尽管随机化检验在原理上具有优势，其缺点也显而易见。首先，对同一数据集的两次分析可能得出不同结论；其次，由于未充分利用全部样本信息，此类检验通常存在功效损失。为克服这些缺陷，本研究探讨如何整合多重随机实现（如通过随机数据分割）产生的检验统计量或p值。我们提出秩变换子抽样作为一种通用方法，在温和假设下为组合统计量或p值提供大样本推断。我们将该方法应用于多类问题，包括高维数据单峰性检验、参数化分位数回归模型拟合优度检验、序贯随机试验中的无直接效应检验，以及交叉拟合双机器学习置信区间的校准。与现有可能高度保守的p值聚合方案不同，本方法能渐近逼近名义水平的I类错误控制。此外，与常规子抽样方法相比，我们证明秩变换可消除备择假设下零分布近似的一阶偏倚，并显著提升检验功效。