Early screening via colonoscopy is critical for colon cancer prevention, yet developing robust AI systems for this domain is hindered by the lack of densely annotated, long-sequence video datasets. Existing datasets predominantly focus on single-class polyp detection and lack the rich spatial, temporal, and linguistic annotations required to evaluate modern Multimodal Large Language Models (MLLMs). To address this critical gap, we introduce Colon-Bench, generated via a novel multi-stage agentic workflow. Our pipeline seamlessly integrates temporal proposals, bounding-box tracking, AI-driven visual confirmation, and human-in-the-loop review to scalably annotate full-procedure videos. The resulting verified benchmark is unprecedented in scope, encompassing 528 videos, 14 distinct lesion categories (including polyps, ulcers, and bleeding), over 300,000 bounding boxes, 213,000 segmentation masks, and 133,000 words of clinical descriptions. We utilize Colon-Bench to rigorously evaluate state-of-the-art MLLMs across lesion classification, Open-Vocabulary Video Object Segmentation (OV-VOS), and video Visual Question Answering (VQA). The MLLM results demonstrate surprisingly high localization performance in medical domains compared to SAM-3. Finally, we analyze common VQA errors from MLLMs to introduce a novel "colon-skill" prompting strategy, improving zero-shot MLLM performance by up to 9.7% across most MLLMs. The dataset and the code are available at https://abdullahamdi.com/colon-bench .

翻译：结肠镜早期筛查对于预防结肠癌至关重要，然而该领域鲁棒人工智能系统的研发因缺乏密集标注的长序列视频数据集而受阻。现有数据集主要聚焦于单类别息肉检测，缺乏评估现代多模态大语言模型（MLLMs）所需的丰富空间、时序及语言标注。为填补这一关键空白，我们提出结肠基准（Colon-Bench），该数据集通过一种新颖的多阶段智能体工作流生成。我们的流水线无缝整合了时序提议、边界框追踪、人工智能驱动的视觉确认及人在环审核，以实现全流程视频的可扩展标注。所得已验证基准在规模上前所未有，涵盖528段视频、14种病灶类别（包括息肉、溃疡和出血）、超过30万个边界框、21.3万个分割掩膜以及13.3万词临床描述。我们利用结肠基准（Colon-Bench）在病灶分类、开放词汇视频目标分割（OV-VOS）及视频视觉问答（VQA）任务上严格评估了最先进的MLLMs。结果显示，与SAM-3相比，MLLMs在医学领域展现了令人惊讶的高定位性能。最后，我们通过分析MLLMs常见的VQA错误，提出了一种新颖的"结肠技能"提示策略，将多数MLLMs的零样本性能提升最高达9.7%。数据集与代码见 https://abdullahamdi.com/colon-bench 。