Large Language Models (LLMs) have demonstrated remarkable proficiency in general medical domains. However, their performance significantly degrades in specialized, culturally specific domains such as Vietnamese Traditional Medicine (VTM), primarily due to the scarcity of high-quality, structured benchmarks. In this paper, we introduce VietMed-MCQ, a novel multiple-choice question dataset generated via a Retrieval-Augmented Generation (RAG) pipeline with an automated consistency check mechanism. Unlike previous synthetic datasets, our framework incorporates a dual-model validation approach to ensure reasoning consistency through independent answer verification, though the substring-based evidence checking has known limitations. The complete dataset of 3,190 questions spans three difficulty levels and underwent validation by one medical expert and four students, achieving 94.2 percent approval with substantial inter-rater agreement (Fleiss' kappa = 0.82). We benchmark seven open-source models on VietMed-MCQ. Results reveal that general-purpose models with strong Chinese priors outperform Vietnamese-centric models, highlighting cross-lingual conceptual transfer, while all models still struggle with complex diagnostic reasoning. Our code and dataset are publicly available to foster research in low-resource medical domains.

翻译：大语言模型在通用医学领域已展现出卓越的能力。然而，在越南传统医学这类专业化、文化特定的领域中，其性能显著下降，这主要源于高质量、结构化基准数据的稀缺。本文介绍了VietMed-MCQ，这是一个通过检索增强生成（RAG）流程结合自动一致性检查机制生成的新型多项选择题数据集。与以往的合成数据集不同，我们的框架采用双模型验证方法，通过独立答案验证来确保推理一致性，尽管基于子字符串的证据检查存在已知的局限性。完整的3,190道问题数据集涵盖三个难度级别，并经过一位医学专家和四名学生的验证，获得了94.2%的认可率，且评分者间一致性较高（Fleiss' kappa = 0.82）。我们在VietMed-MCQ上对七个开源模型进行了基准测试。结果表明，具有强大中文先验知识的通用模型优于以越南语为中心的模型，这凸显了跨语言概念迁移的作用，而所有模型在复杂的诊断推理方面仍存在困难。我们的代码和数据集已公开，以促进低资源医学领域的研究。