Agentic AI "scientists" now use language models to search the literature, run analyses, and generate hypotheses. We evaluate KOSMOS, an autonomous AI scientist, on three problems in radiation biology using simple random-gene null benchmarks. Hypothesis 1: baseline DNA damage response (DDR) capacity across cell lines predicts the p53 transcriptional response after irradiation (GSE30240). Hypothesis 2: baseline expression of OGT and CDO1 predicts the strength of repressed and induced radiation-response modules in breast cancer cells (GSE59732). Hypothesis 3: a 12-gene expression signature predicts biochemical recurrence-free survival after prostate radiotherapy plus androgen deprivation therapy (GSE116918). The DDR-p53 hypothesis was not supported: DDR score and p53 response were weakly negatively correlated (Spearman rho = -0.40, p = 0.76), indistinguishable from random five-gene scores. OGT showed only a weak association (r = 0.23, p = 0.34), whereas CDO1 was a clear outlier (r = 0.70, empirical p = 0.0039). The 12-gene signature achieved a concordance index of 0.61 (p = 0.017) but a non-unique effect size. Overall, KOSMOS produced one well-supported discovery, one plausible but uncertain result, and one false hypothesis, illustrating that AI scientists can generate useful ideas but require rigorous auditing against appropriate null models.

翻译：具备自主能力的AI“科学家”现已能够利用语言模型检索文献、运行分析并生成假设。本研究通过简单的随机基因零假设基准，评估了自主AI科学家KOSMOS在辐射生物学中的三个问题。假设1：跨细胞系的基线DNA损伤反应（DDR）能力可预测辐照后的p53转录反应（数据集GSE30240）。假设2：OGT与CDO1的基线表达水平可预测乳腺癌细胞中受抑制与诱导的辐射反应模块强度（数据集GSE59732）。假设3：一个12基因表达特征可预测前列腺放疗联合雄激素剥夺治疗后的生化无复发生存期（数据集GSE116918）。DDR-p53假设未获支持：DDR评分与p53反应呈弱负相关（Spearman rho = -0.40, p = 0.76），与随机五基因评分无显著差异。OGT仅呈现弱关联（r = 0.23, p = 0.34），而CDO1为明显异常值（r = 0.70, 经验p值 = 0.0039）。12基因特征的一致性指数为0.61（p = 0.017），但效应量不具备唯一性。总体而言，KOSMOS产生了一项强证据发现、一项合理但不确定的结果以及一项错误假设，表明AI科学家能够生成有用思路，但需通过适当的零模型进行严格验证。