With the rapid advances of large language models, it becomes increasingly important to systematically evaluate their multilingual and multicultural capabilities. Previous cultural evaluation benchmarks focus mainly on basic cultural knowledge that can be encoded in linguistic form. Here, we propose SommBench, a multilingual benchmark to assess sommelier expertise, a domain deeply grounded in the senses of smell and taste. While language models learn about sensory properties exclusively through textual descriptions, SommBench tests whether this textual grounding is sufficient to emulate expert-level sensory judgment. SommBench comprises three main tasks: Wine Theory Question Answering (WTQA), Wine Feature Completion (WFC), and Food-Wine Pairing (FWP). SommBench is available in multiple languages: English, Slovak, Swedish, Finnish, German, Danish, Italian, and Spanish. This helps separate a language model's wine expertise from its language skills. The benchmark datasets were developed in close collaboration with a professional sommelier and native speakers of the respective languages, resulting in 1,024 wine theory question-answering questions, 1,000 wine feature-completion examples, and 1,000 food-wine pairing examples. We provide results for the most popular language models, including closed-weights models such as Gemini 2.5, and open-weights models, such as GPT-OSS and Qwen 3. Our results show that the most capable models perform well on wine theory question answering (up to 97% correct with a closed-weights model), yet feature completion (peaking at 65%) and food-wine pairing show (MCC ranging between 0 and 0.39) turn out to be more challenging. These results position SommBench as an interesting and challenging benchmark for evaluating the sommelier expertise of language models. The benchmark is publicly available at https://github.com/sommify/sommbench.

翻译：随着大型语言模型的快速发展，系统评估其多语言与跨文化能力变得日益重要。现有的文化评估基准主要关注可编码为语言形式的基础文化知识。本文提出SommBench，一个用于评估侍酒师专业知识的跨语言基准，该领域深度植根于嗅觉与味觉感官体验。虽然语言模型仅通过文本描述学习感官属性，但SommBench旨在检验这种文本基础是否足以模拟专家级的感官判断。SommBench包含三大任务：葡萄酒理论问答（WTQA）、葡萄酒特征补全（WFC）以及餐酒搭配（FWP）。该基准支持多语言版本：英语、斯洛伐克语、瑞典语、芬兰语、德语、丹麦语、意大利语和西班牙语。这有助于区分语言模型的葡萄酒专业知识与其语言技能。基准数据集由专业侍酒师及各语言母语者协同开发，最终包含1,024道葡萄酒理论问答题目、1,000个葡萄酒特征补全示例和1,000个餐酒搭配示例。我们提供了主流语言模型的测试结果，包括闭源模型（如Gemini 2.5）和开源模型（如GPT-OSS与Qwen 3）。结果表明，性能最强的模型在葡萄酒理论问答任务中表现优异（闭源模型正确率最高达97%），但特征补全（峰值65%）与餐酒搭配（马修斯相关系数介于0至0.39之间）任务更具挑战性。这些结果确立了SommBench作为评估语言模型侍酒师专业知识的独特且具有挑战性的基准地位。该基准已公开于https://github.com/sommify/sommbench。