It has been revealed that small efficient dense image prediction (EDIP) models, trained using the knowledge distillation (KD) framework, encounter two key challenges, including maintaining boundary region completeness and preserving target region connectivity, despite their favorable capacity to recognize main object regions. In this work, we propose a complementary boundary and context distillation (BCD) method within the KD framework for EDIPs, which facilitates the targeted knowledge transfer from large accurate teacher models to compact efficient student models. Specifically, the boundary distillation component focuses on extracting explicit object-level semantic boundaries from the hierarchical feature maps of the backbone network to enhance the student model's mask quality in boundary regions. Concurrently, the context distillation component leverages self-relations as a bridge to transfer implicit pixel-level contexts from the teacher model to the student model, ensuring strong connectivity in target regions. Our proposed BCD method is specifically designed for EDIP tasks and is characterized by its simplicity and efficiency. Extensive experimental results across semantic segmentation, object detection, and instance segmentation on various representative datasets demonstrate that our method can outperform existing methods without requiring extra supervisions or incurring increased inference costs, resulting in well-defined object boundaries and smooth connecting regions.

翻译：研究表明，尽管小型高效密集图像预测（EDIP）模型在识别主要物体区域方面表现出良好的能力，但在基于知识蒸馏（KD）框架训练时仍面临两大挑战：保持边界区域完整性以及维持目标区域连通性。本文提出一种适用于EDIP的互补边界与上下文蒸馏（BCD）方法，该方法在KD框架内实现从大型精确教师模型到紧凑高效学生模型的定向知识迁移。具体而言，边界蒸馏组件专注于从骨干网络的层次化特征图中提取显式的物体级语义边界，以提升学生模型在边界区域的掩码质量；同时，上下文蒸馏组件通过自关联关系作为桥梁，将教师模型中隐式的像素级上下文信息迁移至学生模型，从而确保目标区域的强连通性。我们提出的BCD方法专为EDIP任务设计，具有结构简洁与计算高效的特点。在语义分割、目标检测和实例分割任务上，基于多个代表性数据集的大量实验结果表明：该方法无需额外监督或增加推理成本，即可超越现有方法，生成边界清晰且连通区域平滑的预测结果。