A fundamental assumption of classical hypothesis testing is that the significance threshold $\alpha$ is chosen independently from the data. The validity of confidence intervals likewise relies on choosing $\alpha$ beforehand. We point out that the independence of $\alpha$ is guaranteed in practice because, in most fields, there exists one standard $\alpha$ that everyone uses -- so that $\alpha$ is automatically independent of everything. However, there have been recent calls to decrease $\alpha$ from $0.05$ to $0.005$. We note that this may lead to multiple accepted standard thresholds within one scientific field. For example, different journals may require different significance thresholds. As a consequence, some researchers may be tempted to conveniently choose their $\alpha$ based on their p-value. We use examples to illustrate that this severely invalidates hypothesis tests, and mention some potential solutions.

翻译：经典假设检验的一个基本前提是显著性阈值α的选取独立于数据。置信区间的有效性同样依赖于事先选定α。我们指出，在实践中α的独立性之所以得到保证，是因为在大多数学科领域存在一个被普遍采用的标准α值——这使得α自动独立于一切数据。然而，近期学界出现了将α从0.05降低至0.005的呼声。我们注意到这可能导致同一科学领域内出现多个被接受的标准阈值。例如，不同期刊可能要求不同的显著性阈值。其结果是，部分研究者可能倾向于根据其p值来便利地选择α。我们通过实例说明这种做法会严重破坏假设检验的有效性，并探讨若干潜在的解决方案。