Mathematics is a limited component of solutions to real-world problems, as it expresses only what is expected to be true if all our assumptions are correct, including implicit assumptions that are omnipresent and often incorrect. Statistical methods are rife with implicit assumptions whose violation can be life-threatening when results from them are used to set policy. Among them are that there is human equipoise or unbiasedness in data generation, management, analysis, and reporting. These assumptions correspond to levels of cooperation, competence, neutrality, and integrity that are absent more often than we would like to believe. Given this harsh reality, we should ask what meaning, if any, we can assign to the P-values, 'statistical significance' declarations, 'confidence' intervals, and posterior probabilities that are used to decide what and how to present (or spin) discussions of analyzed data. By themselves, P-values and CI do not test any hypothesis, nor do they measure the significance of results or the confidence we should have in them. The sense otherwise is an ongoing cultural error perpetuated by large segments of the statistical and research community via misleading terminology. So-called 'inferential' statistics can only become contextually interpretable when derived explicitly from causal stories about the real data generator (such as randomization), and can only become reliable when those stories are based on valid and public documentation of the physical mechanisms that generated the data. Absent these assurances, traditional interpretations of statistical results become pernicious fictions that need to be replaced by far more circumspect descriptions of data and model relations.
翻译:数学是解决现实世界问题的有限组成部分,因为它仅表达了在假设正确时(包括普遍存在且常不正确的隐含假设)预期成立的内容。统计方法充满隐含假设,当依据其结果制定政策时,这些假设的违反可能危及生命。其中关键假设包括数据生成、管理、分析和报告过程中存在人类均衡性或无偏性。这些假设对应着合作、能力、中立和诚信的层次,其缺失程度往往超出我们愿意相信的范围。面对这一严酷现实,我们应当自问:用于决定数据分析讨论内容与呈现方式(或误导性解读)的P值、"统计显著性"声明、"置信"区间和后验概率,究竟能赋予何种意义?P值和置信区间本身既不检验任何假设,也不衡量结果的显著性或其应被赋予的可信度。与之相反的理解,是统计界和学术界大量群体通过误导性术语延续的文化谬误。所谓"推断"统计唯有当明确基于真实数据生成机制(如随机化)的因果故事时,才能获得情境化解释;唯有当这些故事建立在生成数据的物理机制的有效公开文档基础上时,才能变得可靠。缺乏这些保障,对统计结果的传统解读将沦为有害的虚构,需代之以对数据与模型关系更为审慎的描述。