The lightweight MLP-based decoder has become increasingly promising for semantic segmentation. However, the channel-wise MLP cannot expand the receptive fields, lacking the context modeling capacity, which is critical to semantic segmentation. In this paper, we propose a parametric-free patch rotate operation to reorganize the pixels spatially. It first divides the feature map into multiple groups and then rotates the patches within each group. Based on the proposed patch rotate operation, we design a novel segmentation network, named PRSeg, which includes an off-the-shelf backbone and a lightweight Patch Rotate MLP decoder containing multiple Dynamic Patch Rotate Blocks (DPR-Blocks). In each DPR-Block, the fully connected layer is performed following a Patch Rotate Module (PRM) to exchange spatial information between pixels. Specifically, in PRM, the feature map is first split into the reserved part and rotated part along the channel dimension according to the predicted probability of the Dynamic Channel Selection Module (DCSM), and our proposed patch rotate operation is only performed on the rotated part. Extensive experiments on ADE20K, Cityscapes and COCO-Stuff 10K datasets prove the effectiveness of our approach. We expect that our PRSeg can promote the development of MLP-based decoder in semantic segmentation.

翻译：基于轻量级MLP的解码器在语义分割中日益展现出潜力。然而，通道维度的MLP无法扩展感受野，缺乏对语义分割至关重要的上下文建模能力。本文提出一种无参数的补丁旋转操作来在空间上重新组织像素。该操作首先将特征图划分为多个组，然后在每组内对补丁进行旋转。基于所提出的补丁旋转操作，我们设计了一种名为PRSeg的新型分割网络，包含现成的骨干网络和轻量级补丁旋转MLP解码器，其中包含多个动态补丁旋转模块（DPR-Blocks）。在每个DPR-Block中，全连接层在补丁旋转模块（PRM）之后执行，以交换像素间的空间信息。具体而言，在PRM中，根据动态通道选择模块（DCSM）预测的概率，特征图首先沿通道维度分为保留部分和旋转部分，而所提出的补丁旋转操作仅作用于旋转部分。在ADE20K、Cityscapes和COCO-Stuff 10K数据集上的大量实验证明了我们方法的有效性。我们期望PRSeg能够推动语义分割中基于MLP解码器的发展。