Most existing ultra-high resolution (UHR) segmentation methods always struggle in the dilemma of balancing memory cost and local characterization accuracy, which are both taken into account in our proposed Guided Patch-Grouping Wavelet Transformer (GPWFormer) that achieves impressive performances. In this work, GPWFormer is a Transformer ($\mathcal{T}$)-CNN ($\mathcal{C}$) mutual leaning framework, where $\mathcal{T}$ takes the whole UHR image as input and harvests both local details and fine-grained long-range contextual dependencies, while $\mathcal{C}$ takes downsampled image as input for learning the category-wise deep context. For the sake of high inference speed and low computation complexity, $\mathcal{T}$ partitions the original UHR image into patches and groups them dynamically, then learns the low-level local details with the lightweight multi-head Wavelet Transformer (WFormer) network. Meanwhile, the fine-grained long-range contextual dependencies are also captured during this process, since patches that are far away in the spatial domain can also be assigned to the same group. In addition, masks produced by $\mathcal{C}$ are utilized to guide the patch grouping process, providing a heuristics decision. Moreover, the congruence constraints between the two branches are also exploited to maintain the spatial consistency among the patches. Overall, we stack the multi-stage process in a pyramid way. Experiments show that GPWFormer outperforms the existing methods with significant improvements on five benchmark datasets.

翻译：现有大多数超高分辨率（UHR）分割方法始终在内存消耗与局部特征刻画精度之间面临两难困境，而我们提出的导向性补丁分组小波变换器（GPWFormer）兼顾了这两方面因素，实现了令人瞩目的性能表现。本文中，GPWFormer是一个Transformer（$\mathcal{T}$）-CNN（$\mathcal{C}$）相互学习框架，其中$\mathcal{T}$以完整UHR图像为输入，同时获取局部细节与细粒度长程上下文依赖关系，而$\mathcal{C}$以下采样图像为输入，学习类别级别的深层上下文。为实现高推理速度与低计算复杂度，$\mathcal{T}$将原始UHR图像分割为多个补丁并动态分组，随后通过轻量级多头小波变换器（WFormer）网络学习低级局部细节。在此过程中，由于空间域中相距较远的补丁可被分配至同一分组，细粒度长程上下文依赖关系亦得以捕获。此外，$\mathcal{C}$生成的掩码被用于引导补丁分组过程，提供启发式决策。同时，利用两条分支间的空间一致性约束来维持补丁间的空间连贯性。总体而言，我们以金字塔方式堆叠多阶段处理过程。实验表明，GPWFormer在五个基准数据集上均显著优于现有方法。