Multi-object multi-part scene segmentation is a challenging task whose complexity scales exponentially with part granularity and number of scene objects. To address the task, we propose a plug-and-play approach termed OLAF. First, we augment the input (RGB) with channels containing object-based structural cues (fg/bg mask, boundary edge mask). We propose a weight adaptation technique which enables regular (RGB) pre-trained models to process the augmented (5-channel) input in a stable manner during optimization. In addition, we introduce an encoder module termed LDF to provide low-level dense feature guidance. This assists segmentation, particularly for smaller parts. OLAF enables significant mIoU gains of $\mathbf{3.3}$ (Pascal-Parts-58), $\mathbf{3.5}$ (Pascal-Parts-108) over the SOTA model. On the most challenging variant (Pascal-Parts-201), the gain is $\mathbf{4.0}$. Experimentally, we show that OLAF's broad applicability enables gains across multiple architectures (CNN, U-Net, Transformer) and datasets. The code is available at olafseg.github.io
翻译:多物体多部件场景分割是一项具有挑战性的任务,其复杂度随部件粒度和场景物体数量的增加呈指数级增长。为解决此任务,我们提出了一种名为OLAF的即插即用方法。首先,我们通过添加包含基于物体的结构线索(前景/背景掩码、边界边缘掩码)的通道来增强输入(RGB)。我们提出了一种权重自适应技术,使常规(RGB)预训练模型能够在优化过程中以稳定的方式处理增强后的(五通道)输入。此外,我们引入了一个名为LDF的编码器模块,以提供低层密集特征引导。这有助于分割,特别是对于较小部件。OLAF在Pascal-Parts-58和Pascal-Parts-108数据集上分别实现了相对于当前最优模型$\mathbf{3.3}$和$\mathbf{3.5}$的显著mIoU提升。在最具挑战性的变体(Pascal-Parts-201)上,提升幅度达到$\mathbf{4.0}$。实验表明,OLAF的广泛适用性使其能够在多种架构(CNN、U-Net、Transformer)和数据集上带来性能增益。代码可在olafseg.github.io获取。