Recent advances in radiology report generation (RRG) have been driven by large paired image-text datasets; however, progress in neuro-oncology has been limited due to a lack of open paired image-report datasets. Here, we introduce BTReport, an open-source framework for brain tumor RRG that constructs natural language radiology reports using deterministically extracted imaging features. Unlike existing approaches that rely on large general-purpose or fine-tuned vision-language models for both image interpretation and report composition, BTReport performs deterministic feature extraction for image analysis and uses large language models only for syntactic structuring and narrative formatting. By separating RRG into a deterministic feature extraction step and a report generation step, the generated reports are completely interpretable and less prone to hallucinations. We show that the features used for report generation are predictive of key clinical outcomes, including survival and IDH mutation status, and reports generated by BTReport are more closely aligned with reference clinical reports than existing baselines for RRG. Finally, we introduce BTReport-BraTS, a companion dataset that augments BraTS imaging with synthetically generated radiology reports produced with BTReport. Code for this project can be found at https://github.com/KurtLabUW/BTReport.
翻译:近年来,放射学报告生成(RRG)的进展主要得益于大型配对图像-文本数据集;然而,由于缺乏开放的配对图像-报告数据集,神经肿瘤学领域的进展一直受限。本文介绍BTReport,这是一个用于脑肿瘤RRG的开源框架,它利用确定性提取的影像特征构建自然语言放射学报告。与现有方法依赖大型通用或微调的视觉-语言模型进行图像解读和报告撰写不同,BTReport采用确定性特征提取进行图像分析,并仅使用大语言模型进行句法结构和叙事格式化。通过将RRG分解为确定性特征提取步骤和报告生成步骤,生成的报告完全可解释且不易产生幻觉。我们证明,用于报告生成的特征能够预测关键的临床结局,包括生存期和IDH突变状态,并且BTReport生成的报告比现有的RRG基线模型更接近参考临床报告。最后,我们介绍了BTReport-BraTS,这是一个配套数据集,它通过BTReport生成的合成放射学报告对BraTS影像进行了增强。本项目的代码可在 https://github.com/KurtLabUW/BTReport 找到。