Superpixels are a useful representation to reduce the complexity of image data. However, to combine superpixels with convolutional neural networks (CNNs) in an end-to-end fashion, one requires extra models to generate superpixels and special operations such as graph convolution. In this paper, we propose a way to implicitly integrate a superpixel scheme into CNNs, which makes it easy to use superpixels with CNNs in an end-to-end fashion. Our proposed method hierarchically groups pixels at downsampling layers and generates superpixels. Our method can be plugged into many existing architectures without a change in their feed-forward path because our method does not use superpixels in the feed-forward path but use them to recover the lost resolution instead of bilinear upsampling. As a result, our method preserves detailed information such as object boundaries in the form of superpixels even when the model contains downsampling layers. We evaluate our method on several tasks such as semantic segmentation, superpixel segmentation, and monocular depth estimation, and confirm that it speeds up modern architectures and/or improves their prediction accuracy in these tasks.

翻译：超像素是一种用于降低图像数据复杂性的有效表示方法。然而，为了以端到端的方式将超像素与卷积神经网络（CNN）结合，需要额外的模型来生成超像素以及图卷积等特殊操作。在本文中，我们提出了一种将超像素方案隐式集成到CNN中的方法，使得超像素能够轻松地与CNN以端到端方式结合使用。我们的方法在下采样层对像素进行层次化分组并生成超像素。由于该方法在前馈路径中不使用超像素，而是用于替代双线性上采样以恢复丢失的分辨率，因此可以轻松插入许多现有架构而无需改变其前馈路径。最终，即使模型包含下采样层，我们的方法也能以超像素形式保留目标边界等细节信息。我们在语义分割、超像素分割和单目深度估计等任务上评估了该方法，并证实其在加速现代架构和/或提升这些任务预测精度方面的有效性。