Clinical decision-making in radiology increasingly benefits from artificial intelligence (AI), particularly through large language models (LLMs). However, traditional retrieval-augmented generation (RAG) systems for radiology question answering (QA) typically rely on single-step retrieval, limiting their ability to handle complex clinical reasoning tasks. Here we propose radiology Retrieval and Reasoning (RaR), a multi-step retrieval and reasoning framework designed to improve diagnostic accuracy, factual consistency, and clinical reliability of LLMs in radiology question answering. We evaluated 25 LLMs spanning diverse architectures, parameter scales (0.5B to >670B), and training paradigms (general-purpose, reasoning-optimized, clinically fine-tuned), using 104 expert-curated radiology questions from previously established RSNA-RadioQA and ExtendedQA datasets. To assess generalizability, we additionally tested on an unseen internal dataset of 65 real-world radiology board examination questions. RaR significantly improved mean diagnostic accuracy over zero-shot prompting and conventional online RAG. The greatest gains occurred in small-scale models, while very large models (>200B parameters) demonstrated minimal changes (<2% improvement). Additionally, RaR retrieval reduced hallucinations (mean 9.4%) and retrieved clinically relevant context in 46% of cases, substantially aiding factual grounding. Even clinically fine-tuned models showed gains from RaR (e.g., MedGemma-27B), indicating that retrieval remains beneficial despite embedded domain knowledge. These results highlight the potential of RaR to enhance factuality and diagnostic accuracy in radiology QA, warranting future studies to validate their clinical utility. All datasets, code, and the full RaR framework are publicly available to support open research and clinical translation.

翻译：放射学临床决策日益受益于人工智能（AI），尤其是大型语言模型（LLMs）的应用。然而，传统用于放射学问答（QA）的检索增强生成（RAG）系统通常依赖单步检索，限制了其处理复杂临床推理任务的能力。本文提出放射学检索与推理（RaR），一种多步检索与推理框架，旨在提升LLMs在放射学问答中的诊断准确性、事实一致性及临床可靠性。我们评估了涵盖不同架构、参数规模（0.5B至>670B）和训练范式（通用型、推理优化型、临床微调型）的25个LLMs，使用了来自已建立的RSNA-RadioQA和ExtendedQA数据集的104道专家策划的放射学问题。为评估泛化能力，我们还额外在一个未见过的内部数据集（包含65道真实世界放射学委员会考试题目）上进行了测试。RaR相较于零样本提示和传统在线RAG显著提升了平均诊断准确率。最大增益出现在小规模模型中，而超大规模模型（>200B参数）则表现出最小变化（改进<2%）。此外，RaR检索减少了幻觉（平均9.4%），并在46%的案例中检索到临床相关上下文，极大地辅助了事实依据。即使是临床微调模型（如MedGemma-27B）也从RaR中获益，表明尽管具备领域知识，检索仍然有益。这些结果凸显了RaR在提升放射学QA事实性和诊断准确性方面的潜力，值得未来研究验证其临床效用。所有数据集、代码及完整的RaR框架均已公开，以支持开放研究和临床转化。