Many testing problems are readily amenable to randomised tests such as those employing data splitting. However despite their usefulness in principle, randomised tests have obvious drawbacks. Firstly, two analyses of the same dataset may lead to different results. Secondly, the test typically loses power because it does not fully utilise the entire sample. As a remedy to these drawbacks, we study how to combine the test statistics or p-values resulting from multiple random realisations such as through random data splits. We develop rank-transformed subsampling as a general method for delivering large sample inference about the combined statistic or p-value under mild assumptions. We apply our methodology to a wide range of problems, including testing unimodality in high-dimensional data, testing goodness-of-fit of parametric quantile regression models, testing no direct effect in a sequentially randomised trial and calibrating cross-fit double machine learning confidence intervals. In contrast to existing p-value aggregation schemes that can be highly conservative, our method enjoys type-I error control that asymptotically approaches the nominal level. Moreover, compared to using the ordinary subsampling, we show that our rank transform can remove the first-order bias in approximating the null under alternatives and greatly improve power.

翻译：许多检验问题易于采用随机化检验（如数据分割法）处理。然而，尽管随机化检验原则上具有实用性，但其存在明显缺陷。首先，同一数据集的两份分析可能产生不同结果。其次，由于未能充分利用全部样本，检验效能通常会降低。针对这些不足，我们研究了如何整合多次随机实现（如随机数据分割）的检验统计量或p值。我们发展了秩变换子抽样这一通用方法，可在温和假设下对组合统计量或p值进行大样本推断。我们将该方法应用于广泛问题，包括高维数据单峰性检验、参数分位数回归模型拟合优度检验、序贯随机试验中直接效应缺失检验，以及交叉拟合双重机器学习置信区间的校准。与现有可能高度保守的p值聚合方案相比，我们的方法在渐近水平上实现了I类错误控制。此外，与普通子抽样相比，我们证明了秩变换能在备择假设下消除近似零分布的一阶偏差，并显著提升检验功效。