The development of practical (multimodal) large language model assistants for Korean weather forecasters is hindered by the absence of a multidimensional, expert-level evaluation framework grounded in authoritative sources. To address this, we introduce K-MetBench, a diagnostic benchmark grounded in national qualification exams. It exposes critical gaps across four dimensions: expert visual reasoning of charts, logical validity via expert-verified rationales, Korean-specific geo-cultural comprehension, and fine-grained domain analysis. Our evaluation of 55 models reveals a profound modality gap in interpreting specialized diagrams and a reasoning gap where models hallucinate logic despite correct predictions. Crucially, Korean models outperform significantly larger global models in local contexts, demonstrating that parameter scaling alone cannot resolve cultural dependencies. K-MetBench serves as a roadmap for developing reliable, culturally aware expert AI agents. The dataset is available at https://huggingface.co/datasets/soyeonbot/K-MetBench .

翻译：针对韩国气象预报员的实用型（多模态）大语言模型助手的开发，因缺乏基于权威来源的多维度、专家级评估框架而受阻。为此，我们提出K-MetBench——一项基于国家资格考试构建的诊断性基准。该基准从四个维度揭示了关键差距：图表的专家视觉推理、基于专家验证理由的逻辑有效性、韩国特定地理文化理解，以及细粒度领域分析。我们对55个模型的评估发现：在解读专业图表时存在显著的模态差距，且存在推理差距（即模型在预测正确的情况下仍出现逻辑幻觉）。至关重要的是，韩国模型在本地情境中显著优于规模更大的全球模型，这表明仅靠参数扩展无法解决文化依赖性问题。K-MetBench为开发可靠且具文化感知力的专家级AI智能体提供了路线图。数据集可于https://huggingface.co/datasets/soyeonbot/K-MetBench获取。