Detecting weak, systematic signals hidden in a large collection of $p$-values published in academic journals is instrumental to identifying and understanding publication bias and $p$-value hacking in social and economic sciences. Given two probability distributions $P$ (null) and $Q$ (signal), we study the problem of detecting weak signals from the null $P$ based on $n$ independent samples: we model weak signals via displacement interpolation between $P$ and $Q$, where the signal strength vanishes with $n$. We propose a hypothesis testing procedure based on the Wasserstein distance from optimal transport theory, derive sharp conditions under which detection is possible, and provide the exact characterization of the asymptotic Type I and Type II errors at the detection boundary using empirical processes. Applying our testing procedure to real data sets on published $p$-values across academic journals, we demonstrate that a rigorous testing procedure can detect weak signals that are otherwise indistinguishable.

翻译：检测学术期刊大量已发表p值中隐藏的弱系统性信号，对于识别和理解社会科学与经济学中的发表偏倚及p值操纵至关重要。针对两个概率分布P（零假设）与Q（信号），我们研究基于n个独立样本从零假设P中检测弱信号的问题：通过P与Q之间的位移插值对弱信号进行建模，其中信号强度随n增大而衰减。基于最优输运理论中的Wasserstein距离提出假设检验流程，推导出实现检测的临界条件，并利用经验过程在检测边界处给出渐近第一类与第二类误差的精确刻画。将所提检验流程应用于学术期刊已发表p值的真实数据集，结果表明该严格检验流程能够检测出原本难以识别的弱信号。