Despite the significant success of deep learning in computer vision tasks, cross-domain tasks still present a challenge in which the model's performance will degrade when the training set and the test set follow different distributions. Most existing methods employ adversarial learning or instance normalization for achieving data augmentation to solve this task. In contrast, considering that the batch normalization (BN) layer may not be robust for unseen domains and there exist the differences between local patches of an image, we propose a novel method called patch-aware batch normalization (PBN). To be specific, we first split feature maps of a batch into non-overlapping patches along the spatial dimension, and then independently normalize each patch to jointly optimize the shared BN parameter at each iteration. By exploiting the differences between local patches of an image, our proposed PBN can effectively enhance the robustness of the model's parameters. Besides, considering the statistics from each patch may be inaccurate due to their smaller size compared to the global feature maps, we incorporate the globally accumulated statistics with the statistics from each batch to obtain the final statistics for normalizing each patch. Since the proposed PBN can replace the typical BN, it can be integrated into most existing state-of-the-art methods. Extensive experiments and analysis demonstrate the effectiveness of our PBN in multiple computer vision tasks, including classification, object detection, instance retrieval, and semantic segmentation.

翻译：尽管深度学习在计算机视觉任务中取得了显著成功，但跨域任务仍是一个挑战：当训练集和测试集遵循不同分布时，模型性能会下降。现有方法多采用对抗学习或实例归一化来实现数据增强以解决此问题。与此不同，考虑到批量归一化（BN）层可能对未见域不够鲁棒，且图像局部补丁间存在差异，我们提出一种新方法——面向补丁的批量归一化（PBN）。具体而言，我们首先将一批特征图沿空间维度分割为不重叠的补丁，然后独立归一化每个补丁，并在每次迭代中联合优化共享的BN参数。通过利用图像局部补丁间的差异，所提PBN能有效增强模型参数的鲁棒性。此外，考虑到每个补丁因尺寸小于全局特征图可能导致其统计量不准确，我们将全局累积的统计量与每批统计量结合，得到归一化每个补丁的最终统计量。由于所提PBN可替代标准BN，它能集成到大多数现有最优方法中。大量实验和分析表明，PBN在分类、目标检测、实例检索和语义分割等多项计算机视觉任务中具有有效性。