Recent advancements in video models have shown tremendous progress, particularly in long video understanding. However, current benchmarks predominantly feature western-centric data and English as the dominant language, introducing significant biases in evaluation. To address this, we introduce CURVE (Cultural Understanding and Reasoning in Video Evaluation), a challenging benchmark for multicultural and multilingual video reasoning. CURVE comprises high-quality, entirely human-generated annotations from diverse, region-specific cultural videos across 18 global locales. Unlike prior work that relies on automatic translations, CURVE provides complex questions, answers, and multi-step reasoning steps, all crafted in native languages. Making progress on CURVE requires a deeply situated understanding of visual cultural context. Furthermore, we leverage CURVE's reasoning traces to construct evidence-based graphs and propose a novel iterative strategy using these graphs to identify fine-grained errors in reasoning. Our evaluations reveal that SoTA Video-LLMs struggle significantly, performing substantially below human-level accuracy, with errors primarily stemming from the visual perception of cultural elements. CURVE will be publicly available under https://github.com/google-deepmind/neptune?tab=readme-ov-file\#minerva-cultural

翻译：近期视频模型取得了显著进展，尤其在长视频理解方面。然而，现有基准数据集主要包含以西方为中心的数据且以英语为主导语言，导致评估存在显著偏差。为此，我们提出了CURVE（视频评估中的文化理解与推理基准），这是一个面向多元文化与多语言视频推理的挑战性基准。CURVE包含来自全球18个地区多样化本土文化视频的高质量、完全人工生成的标注。与以往依赖自动翻译的研究不同，CURVE提供了以原生语言精心构建的复杂问题、答案及多步推理过程。要在CURVE上取得进展，需要对视觉文化背景进行深度情境化理解。此外，我们利用CURVE的推理轨迹构建了基于证据的图结构，并提出一种新颖的迭代策略，通过该图识别推理过程中的细粒度错误。评估结果表明，当前最先进的视频大语言模型表现严重不足，其准确率远低于人类水平，错误主要源于对文化元素的视觉感知不足。CURVE将通过https://github.com/google-deepmind/neptune?tab=readme-ov-file#minerva-cultural 公开提供。