Recent advancements in generative AI have flourished the development of highly adept Large Language Models (LLMs) that integrate diverse data types to empower decision-making. Among these, Multimodal Retrieval-Augmented Generation (RAG) applications are promising for their capability to combine the strengths of information retrieval and generative models, enhancing their utility across various domains, including biomedical research. This paper introduces AlzheimerRAG, a Multimodal RAG pipeline tool for biomedical research use cases, primarily focusing on Alzheimer's disease from PubMed articles. Our pipeline incorporates multimodal fusion techniques to integrate textual and visual data processing by efficiently indexing and accessing vast amounts of biomedical literature. Preliminary experimental results against benchmarks, such as BioASQ and PubMedQA, have returned improved results in information retrieval and synthesis of domain-specific information. We also demonstrate a case study with our RAG pipeline across different Alzheimer's clinical scenarios. We infer that AlzheimerRAG can generate responses with accuracy non-inferior to humans and with low rates of hallucination. Overall, a reduction in cognitive task load is observed, which allows researchers to gain multimodal insights, improving understanding and treatment of Alzheimer's disease.

翻译：近期生成式人工智能的进展推动了高度熟练的大型语言模型（LLM）的发展，这些模型通过整合多模态数据类型来增强决策能力。其中，多模态检索增强生成（RAG）应用展现出巨大潜力，因其能够结合信息检索与生成模型的优势，从而提升在生物医学研究等领域的实用性。本文介绍AlzheimerRAG——一个面向生物医学研究用例的多模态RAG流程工具，主要聚焦于PubMed文献中关于阿尔茨海默病的研究。该流程采用多模态融合技术，通过高效索引与访问海量生物医学文献，实现对文本与视觉数据的协同处理。在BioASQ和PubMedQA等基准测试中的初步实验结果表明，该系统在领域特定信息的检索与合成方面取得了改进效果。我们通过不同阿尔茨海默病临床场景的案例研究展示了该RAG流程的应用。实验推断AlzheimerRAG生成的响应在准确性上不亚于人类专家，且具有较低的幻觉产生率。总体而言，该系统可显著降低研究人员的认知任务负荷，帮助其获得多维度洞察，从而提升对阿尔茨海默病的理解与治疗水平。