Search agents are language models (LMs) that reason and search knowledge bases (or the web) to answer questions; recent methods supervise only the final answer accuracy using reinforcement learning with verifiable rewards (RLVR). Most RLVR search agents tackle general-domain QA, which limits their relevance to technical AI systems in science, engineering, and medicine. In this work we propose training agents to search and reason over scientific papers -- this tests technical question-answering, it is directly relevant to real scientists, and the capabilities will be crucial to future AI Scientist systems. Concretely, we release a search corpus of 16 million biomedical paper abstracts and construct a challenging factoid QA dataset called PaperSearchQA with 60k samples answerable from the corpus, along with benchmarks. We train search agents in this environment to outperform non-RL retrieval baselines; we also perform further quantitative analysis and observe interesting agent behaviors like planning, reasoning, and self-verification. Our corpus, datasets, and benchmarks are usable with the popular Search-R1 codebase for RLVR training and released on https://huggingface.co/collections/jmhb/papersearchqa. Finally, our data creation methods are scalable and easily extendable to other scientific domains.

翻译：搜索代理是通过推理和搜索知识库（或网络）来回答问题的语言模型；近期方法仅使用可验证奖励强化学习监督最终答案的准确性。大多数RLVR搜索代理处理通用领域问答，这限制了其在科学、工程和医学领域技术人工智能系统中的适用性。本工作提出训练代理在科研论文中进行搜索与推理——这既能检验技术问答能力，又直接关联实际科研需求，相关能力对未来AI科学家系统至关重要。具体而言，我们发布了包含1600万篇生物医学论文摘要的搜索语料库，并构建了包含6万个可回答样本的挑战性事实型问答数据集PaperSearchQA及基准测试。在此环境中训练的搜索代理性能优于非强化学习检索基线；我们进一步开展定量分析，观察到代理表现出规划、推理和自我验证等有趣行为。本研究的语料库、数据集和基准测试可通过流行的RLVR训练代码库Search-R1使用，并发布于https://huggingface.co/collections/jmhb/papersearchqa。最后，我们的数据创建方法具有可扩展性，可轻松延伸至其他科学领域。