Autonomous driving has long faced a challenge with public acceptance due to the lack of explainability in the decision-making process. Video question-answering (QA) in natural language provides the opportunity for bridging this gap. Nonetheless, evaluating the performance of Video QA models has proved particularly tough due to the absence of comprehensive benchmarks. To fill this gap, we introduce LingoQA, a benchmark specifically for autonomous driving Video QA. The LingoQA trainable metric demonstrates a 0.95 Spearman correlation coefficient with human evaluations. We introduce a Video QA dataset of central London consisting of 419k samples that we release with the paper. We establish a baseline vision-language model and run extensive ablation studies to understand its performance.

翻译：自动驾驶因决策过程缺乏可解释性，长期面临公众接受度的挑战。自然语言形式的视频问答（QA）为弥合这一差距提供了可能。然而，由于缺乏全面的基准测试，评估视频问答模型的性能尤为困难。为填补这一空白，我们提出LingoQA——专为自动驾驶视频问答设计的基准测试集。LingoQA可训练的评估指标与人工评估的斯皮尔曼相关系数达到0.95。我们发布了包含41.9万个样本、覆盖伦敦市中心的视频问答数据集。本文建立了一个基线视觉语言模型，并通过广泛的消融实验深入分析其性能表现。