Can humans tell whether a news article was written by a person or a large language model (LLM)? We investigate this question using JudgeGPT, a study platform that independently measures source attribution (human vs. machine) and authenticity judgment (legitimate vs. fake) on continuous scales. From 2,318 judgments collected from 1,054 participants across content generated by six LLMs, we report five findings: (1) participants cannot reliably distinguish machine-generated from human-written text (p > .05, Welch's t-test); (2) this inability holds across all tested models, including open-weight models with as few as 7B parameters; (3) self-reported domain expertise predicts judgment accuracy (r = .35, p < .001) whereas political orientation does not (r = -.10, n.s.); (4) clustering reveals distinct response strategies ("Skeptics" vs. "Believers"); and (5) accuracy degrades after approximately 30 sequential evaluations due to cognitive fatigue. The answer, in short, is no: humans cannot reliably tell. These results indicate that user-side detection is not a viable defense and motivate system-level countermeasures such as cryptographic content provenance.

翻译：人类能否分辨一篇新闻文章是由人还是大型语言模型（LLM）所写？我们通过JudgeGPT这一研究平台对此问题展开探究，该平台独立测量源属性（人类与机器）和真实性判断（合法与虚假）在连续量表上的表现。基于来自1054名参与者对六个LLM生成内容所收集的2318项判断，我们报告五项发现：（1）参与者无法可靠区分机器生成文本与人类撰写文本（p > .05，韦尔奇t检验）；（2）这种无法区分性在所有测试模型中均成立，包括参数低至70亿的开源模型；（3）自我报告的领域专长能预测判断准确率（r = .35，p < .001），而政治倾向则不能（r = -.10，不显著）；（4）聚类分析揭示了不同的响应策略（"怀疑者"与"相信者"）；（5）由于认知疲劳，在大约30次连续评估后准确率下降。简而言之，答案是否定的：人类无法可靠分辨。这些结果表明用户端检测并非可行防御措施，并推动了诸如加密内容溯源等系统级对策。