Many statistical problems can be addressed by applying a multiple testing procedure (MTP) that controls either the Family-wise Error Rate (FWER) or False Discovery Rate (FDR) under unknown arbitrarily-interdependent $p$-values, without explicitly modeling these inter-correlations. They include the FWER-controlling Bonferroni (1936) MTP and Holm (1979) MTP; the FDR-controlling Benjamini and Yekutieli (2001) MTP; and the DP-MTP (Karabatsos, 2025), based on a Dirichlet process (DP) prior distribution supporting the entire space of MTPs that control either the FWER or FDR. For such an MTP, this study introduces a new and congenial method for Bayesian predictive power analysis, for power calculation and sample size determination for any given planned future (e.g., replication or interim) study. This novel MTP predictive power analysis method is based on a joint prior distribution defining a scale matrix mixture of asymmetric multivariate normal mean-variance mixture distributions, factorized as a general prior distribution for effect sizes (e.g., obtained from expert judgment or results of prior studies), and a uniform prior distribution for correlation matrices representing arbitrary dependencies between $p$-values of test statistics of given multiple hypothesis tests under their alternative hypotheses. The new MTP power analysis method also results in $p$-value weights which can be used to minimize the relative impacts of and assess for significance-chasing biases (e.g., publication bias, $p$-hacking, etc.) in multiple testing, without needing to assume that $p$-values (effect sizes) are independent. The new simulation-based MTP predictive power analysis method is illustrated through the analysis of $p$-values obtained by a famous study of lead exposure and re-analyzed by the previous MTP literature, using R package bnpMTP.
翻译:许多统计问题可以通过应用多重检验程序(MTP)来解决,这些程序在未知且任意相互依赖的 $p$ 值下控制族错误率(FWER)或错误发现率(FDR),而无需显式建模这些相互关联。这些方法包括控制FWER的Bonferroni(1936)MTP和Holm(1979)MTP;控制FDR的Benjamini和Yekutieli(2001)MTP;以及基于狄利克雷过程(DP)先验分布的DP-MTP(Karabatsos,2025),该先验分布支持控制FWER或FDR的整个MTP空间。针对此类MTP,本研究引入了一种新颖且协调的贝叶斯预测功效分析方法,用于计算任何给定计划未来(例如,重复或中期)研究的功效并确定样本量。这种新颖的MTP预测功效分析方法基于一个联合先验分布,该分布定义了非对称多元正态均值-方差混合分布的尺度矩阵混合,并分解为效应大小的一般先验分布(例如,从专家判断或先前研究结果获得)和一个表示给定多重假设检验在其备择假设下检验统计量 $p$ 值之间任意依赖关系的相关矩阵的均匀先验分布。这种新的MTP功效分析方法还产生了 $p$ 值权重,可用于最小化多重检验中显著性追逐偏差(例如,发表偏倚、$p$ 值操纵等)的相对影响并对其进行评估,而无需假设 $p$ 值(效应大小)是独立的。这种新的基于模拟的MTP预测功效分析方法通过分析一项著名的铅暴露研究(该研究曾被先前的MTP文献重新分析)获得的 $p$ 值进行了说明,分析使用了R包bnpMTP。