Addressing Out-Of-Distribution (OOD) Segmentation and Zero-Shot Semantic Segmentation (ZS3) is challenging, necessitating segmenting unseen classes. Existing strategies adapt the class-agnostic Mask2Former (CA-M2F) tailored to specific tasks. However, these methods cater to singular tasks, demand training from scratch, and we demonstrate certain deficiencies in CA-M2F, which affect performance. We propose the Class-Agnostic Structure-Constrained Learning (CSL), a plug-in framework that can integrate with existing methods, thereby embedding structural constraints and achieving performance gain, including the unseen, specifically OOD, ZS3, and domain adaptation (DA) tasks. There are two schemes for CSL to integrate with existing methods (1) by distilling knowledge from a base teacher network, enforcing constraints across training and inference phrases, or (2) by leveraging established models to obtain per-pixel distributions without retraining, appending constraints during the inference phase. We propose soft assignment and mask split methodologies that enhance OOD object segmentation. Empirical evaluations demonstrate CSL's prowess in boosting the performance of existing algorithms spanning OOD segmentation, ZS3, and DA segmentation, consistently transcending the state-of-art across all three tasks.

翻译：解决分布外分割与零样本语义分割（ZS3）具有挑战性，需要对未见类别进行分割。现有策略采用针对特定任务调整的类别无关Mask2Former（CA-M2F）方法，但这类方法仅适用于单一任务且需要从头训练。我们揭示了CA-M2F存在的若干缺陷，这些缺陷影响了模型性能。为此提出类别无关结构约束学习（CSL）——一种可嵌入现有方法的即插即用框架，通过嵌入结构约束来提升包含未见类场景下的性能，具体涵盖分布外（OOD）分割、ZS3及域适应（DA）三类任务。CSL集成现有方案有两种途径：（1）通过基础教师网络蒸馏知识，在训练和推理阶段强制执行约束；（2）利用预训练模型获取逐像素分布而无需重新训练，仅在推理阶段附加约束。我们提出软分配和掩码分割方法以增强OOD目标分割性能。实验评估证明，CSL能显著提升涵盖OOD分割、ZS3及DA分割的现有算法性能，在三项任务上均持续超越现有最优水平。