P-hacking is prevalent in reality but absent from classical hypothesis testing theory. As a consequence, significant results are much more common than they are supposed to be when the null hypothesis is in fact true. In this paper, we build a model of hypothesis testing with p-hacking. From the model, we construct critical values such that, if the values are used to determine significance, and if scientists' p-hacking behavior adjusts to the new significance standards, significant results occur with the desired frequency. Such robust critical values allow for p-hacking so they are larger than classical critical values. To illustrate the amount of correction that p-hacking might require, we calibrate the model using evidence from the medical sciences. In the calibrated model the robust critical value for any test statistic is the classical critical value for the same test statistic with one fifth of the significance level.

翻译：p-hacking在实际中普遍存在，但在经典假设检验理论中却被忽视。其后果是，当零假设实际为真时，显著结果的出现频率远超理论预期。本文构建了一个包含p-hacking的假设检验模型。基于该模型，我们构造了临界值，使得若用这些值判定显著性，且科学家的p-hacking行为适应新的显著性标准时，显著结果能以期望频率出现。此类稳健临界值因允许p-hacking存在，故大于经典临界值。为说明p-hacking可能需要的修正幅度，我们利用医学科学领域的证据对模型进行校准。在校准后的模型中，任何检验统计量的稳健临界值，相当于该检验统计量在显著性水平降低至五分之一时的经典临界值。