Multimodal large language models (MLLMs) are increasingly deployed as assistants that interact through text and images, making it crucial to evaluate contextual safety when risk depends on both the visual scene and the evolving dialogue. Existing contextual safety benchmarks are mostly single-turn and often miss how malicious intent can emerge gradually or how the same scene can support both benign and exploitative goals. We introduce the Multi-Turn Multimodal Contextual Safety Benchmark (MTMCS-Bench), a benchmark of realistic images and multi-turn conversations that evaluates contextual safety in MLLMs under two complementary settings, escalation-based risk and context-switch risk. MTMCS-Bench offers paired safe and unsafe dialogues with structured evaluation. It contains over 30 thousand multimodal (image+text) and unimodal (text-only) samples, with metrics that separately measure contextual intent recognition, safety-awareness on unsafe cases, and helpfulness on benign ones. Across eight open-source and seven proprietary MLLMs, we observe persistent trade-offs between contextual safety and utility, with models tending to either miss gradual risks or over-refuse benign dialogues. Finally, we evaluate five current guardrails and find that they mitigate some failures but do not fully resolve multi-turn contextual risks.

翻译：多模态大语言模型正越来越多地作为通过文本和图像进行交互的助手被部署，这使得评估其上下文安全性变得至关重要，因为风险往往同时取决于视觉场景和不断演进的对话。现有的上下文安全基准大多为单轮形式，常常忽略了恶意意图如何逐步显现，或同一场景如何可能同时支持良性和恶意的目标。我们引入了多轮多模态上下文安全基准，这是一个包含真实图像和多轮对话的基准，用于在两种互补的设置下评估多模态大语言模型的上下文安全性：基于风险升级的设置和上下文切换风险的设置。该基准提供了成对的、结构化的安全与不安全对话用于评估。它包含超过三万个多模态样本和单模态样本，并提供了分别衡量上下文意图识别能力、对不安全案例的安全意识以及对良性案例的有用性的指标。在对八个开源模型和七个专有模型的评估中，我们观察到模型在上下文安全性与实用性之间存在持续的权衡，模型往往要么未能识别逐步升级的风险，要么对良性对话过度拒绝。最后，我们评估了五种当前的防护机制，发现它们能缓解部分失败案例，但未能完全解决多轮对话中的上下文风险。