Developing the required technology to assist medical experts in their everyday activities is currently a hot topic in the Artificial Intelligence research field. Thus, a number of large language models (LLMs) and automated benchmarks have recently been proposed with the aim of facilitating information extraction in Evidence-Based Medicine (EBM) using natural language as a tool for mediating in human-AI interaction. The most representative benchmarks are limited to either multiple-choice or long-form answers and are available only in English. In order to address these shortcomings, in this paper we present a new dataset which, unlike previous work: (i) includes not only explanatory arguments for the correct answer, but also arguments to reason why the incorrect answers are not correct; (ii) the explanations are written originally by medical doctors to answer questions from the Spanish Residency Medical Exams. Furthermore, this new benchmark allows us to setup a novel extractive task which consists of identifying the explanation of the correct answer written by medical doctors. An additional benefit of our setting is that we can leverage the extractive QA paradigm to automatically evaluate performance of LLMs without resorting to costly manual evaluation by medical experts. Comprehensive experimentation with language models for Spanish shows that sometimes multilingual models fare better than monolingual ones, even outperforming models which have been adapted to the medical domain. Furthermore, results across the monolingual models are mixed, with supposedly smaller and inferior models performing competitively. In any case, the obtained results show that our novel dataset and approach can be an effective technique to help medical practitioners in identifying relevant evidence-based explanations for medical questions.

翻译：开发辅助医学专家日常活动的必要技术是当前人工智能研究领域的热点。为此，近期提出了大量大型语言模型（LLMs）和自动化基准，旨在利用自然语言作为人机交互中介工具，促进循证医学（EBM）中的信息提取。最具代表性的基准局限于选择题或长文回答，且仅支持英文。为解决这些不足，本文提出了一个新数据集，与以往研究不同的是：（i）该数据集不仅包含正确答案的解释性论证，还包含推理为何错误选项不正确的论证；（ii）解释内容由医学专家原创撰写，用于解答西班牙住院医师考试试题。此外，这一新基准使我们能够设立一项全新的抽取式任务，即识别由医学专家撰写的正确答案解释。本设置的另一优势在于，可借助抽取式问答（extractive QA）范式自动评估LLMs性能，无需耗费高昂的人工医学专家评估。针对西班牙语语言模型的综合实验表明，多语言模型有时优于单语言模型，甚至超越经医学领域适配的模型。此外，单语言模型之间的结果呈现混合态势，假设中规模较小、能力较弱的模型表现出竞争性。无论如何，所得结果表明，我们的新数据集和方法可作为有效技术，协助医学从业者识别与医学问题相关的循证解释。