The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p-hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view. I argue that questionable and other research practices do not usually inflate relevant Type I error rates. I begin with an introduction to Type I error rates that distinguishes them from theoretical errors. I then illustrate my argument with respect to model misspecification, multiple testing, selective inference, forking paths, exploratory analyses, p-hacking, optional stopping, double dipping, and HARKing. In each case, I demonstrate that relevant Type I error rates are not usually inflated above their nominal level, and in the rare cases that they are, the inflation is easily identified and resolved. I conclude that the replication crisis may be explained, at least in part, by researchers' misinterpretation of statistical errors and their underestimation of theoretical errors.
翻译:第一类错误率的膨胀被认为是复制危机的原因之一。诸如p-hacking等可疑研究实践被认为会使得第一类错误率高于其名义水平,从而导致文献中出现意外高水平的假阳性结果,进而引发异常低的复制率。本文提出了一种替代性观点。我认为,可疑及其他研究实践通常不会膨胀相关第一类错误率。我首先介绍了第一类错误率的概念,并将其与理论错误区分开来。随后,我分别针对模型误设、多重检验、选择性推断、分叉路径、探索性分析、p-hacking、可选停止、双重 dipping 以及 HARKing 等情形阐述我的论点。在每种情形中,我均论证了相关第一类错误率通常不会高于名义水平,即便在少数膨胀的案例中,这种膨胀也易于识别和解决。我的结论是,复制危机至少部分可归因于研究者对统计错误的误读以及对理论错误的低估。