Recent reports claim that Large Language Models (LLMs) have achieved the ability to derive new science and exhibit human-level general intelligence. We argue that such claims are not rigorous scientific claims, as they do not satisfy Popper's refutability principle (often termed falsifiability), which requires that scientific statements be capable of being disproven. We identify several methodological pitfalls in current AI research on reasoning, including the inability to verify the novelty of findings due to opaque and non-searchable training data, the lack of reproducibility caused by continuous model updates, and the omission of human-interaction transcripts, which obscures the true source of scientific discovery. Additionally, the absence of counterfactuals and data on failed attempts creates a selection bias that may exaggerate LLM capabilities. To address these challenges, we propose guidelines for scientific transparency and reproducibility for research on reasoning by LLMs. Establishing such guidelines is crucial for both scientific integrity and the ongoing societal debates regarding fair data usage.

翻译：近期有报道声称，大型语言模型（LLMs）已具备推导新科学知识的能力，并展现出人类水平的通用智能。我们认为此类主张并非严谨的科学论断，因其未能满足波普尔的可证伪性原则（常被称为可反驳性），该原则要求科学陈述必须能够被证伪。我们指出了当前人工智能推理研究中存在的若干方法论缺陷，包括：由于训练数据不透明且不可检索导致无法验证发现的新颖性；持续模型更新造成的不可复现性；以及人类交互记录的缺失——这掩盖了科学发现的真实来源。此外，反事实场景与失败尝试数据的缺失造成了选择偏差，可能夸大LLMs的实际能力。为应对这些挑战，我们针对LLMs推理研究提出了科学透明性与可复现性准则。建立此类准则对于维护科学严谨性，以及当前关于数据公平使用的社会讨论都至关重要。