Segment Anything Model 3 (SAM3) advances open-vocabulary segmentation through promptable concept segmentation, enabling users to segment all instances associated with a given concept using short noun-phrase (NP) prompts. While effective for concept-level grounding, real-world interactions often involve far richer natural-language instructions that combine attributes, relations, actions, states, or implicit reasoning. Currently, SAM3 relies on external multi-modal agents to convert complex instructions into NPs and conducts iterative mask filtering, leading to coarse representations and limited instance specificity. In this work, we present SAM3-I, an instruction-following extension of the SAM family that unifies concept-level grounding and instruction-level reasoning within a single segmentation framework. Built upon SAM3, SAM3-I introduces an instruction-aware cascaded adaptation mechanism with dedicated alignment losses that progressively aligns expressive instruction semantics with SAM3's vision-language representations, enabling direct interpretation of natural-language instructions while preserving its strong concept recall ability. To enable instruction-following learning, we introduce HMPL-Instruct, a large-scale instruction-centric dataset that systematically covers hierarchical instruction semantics and diverse target granularities. Experiments demonstrate that SAM3-I achieves appealing performance across referring and reasoning-based segmentation, showing that SAM3 can be effectively extended to follow complex natural-language instructions without sacrificing its original concept-driven strengths. Code and dataset are available at https://github.com/debby-0527/SAM3-I.

翻译：SAM3（Segment Anything Model 3）通过可提示概念分割推动了开放词汇分割的发展，使用户能够利用简短名词短语提示对给定概念的所有实例进行分割。尽管该方法在概念级定位上表现有效，但实际交互往往涉及更丰富的自然语言指令，例如属性、关系、动作、状态或隐含推理的组合。当前，SAM3依赖外部多模态智能体将复杂指令转换为名词短语，并通过迭代式掩膜过滤机制进行处理，导致表征粗糙且实例特异性受限。本文提出SAM3-I——SAM系列的指令跟随扩展版本，该模型在统一分割框架中整合了概念级定位与指令级推理能力。基于SAM3架构，SAM3-I引入指令感知级联适配机制，并配合专用对齐损失函数，渐进式地将表达性指令语义与SAM3的视觉语言表征对齐，在保持其强大概念召回能力的同时，实现对自然语言指令的直接解析。为实现指令跟随学习，我们构建了HMPL-Instruct——一个大规模指令中心数据集，系统化覆盖层级指令语义与多样化目标粒度。实验表明，SAM3-I在指代分割与推理分割任务中均取得优异性能，证明SAM3可被有效扩展以遵循复杂自然语言指令，且不牺牲其原有的概念驱动优势。代码与数据集开源地址：https://github.com/debby-0527/SAM3-I。