To cope with the high requirements during the computation of semantic segmentations of earth observation imagery, current state-of-the-art pipelines divide the corresponding data into smaller images. Existing methods and benchmark datasets oftentimes rely on pixel-based tiling schemes or on geo-tiling schemes employed by web mapping applications. The selection of the subimages (comprising size, location and orientation) is crucial since it affects the available context information of each pixel, defines the number of tiles during training, and influences the degree of information degradation while down- and up-sampling the tile contents to the size required by the segmentation model. In this paper we propose a new segmentation pipeline for earth observation imagery relying on a tiling scheme that creates geo-tiles based on the geo-information of the raster data. This approach exhibits several beneficial properties compared to pixel-based or common web mapping approaches. For instance, the proposed tiling scheme shows flexible customization properties regarding tile granularity, tile stride and image boundary alignment, which allows us to perform a tile specific data augmentation during training and a substitution of pixel predictions with limited context information using data of overlapping tiles during inference. Furthermore, the generated tiles show a consistent spatial tile extent w.r.t. heterogeneous sensors, varying recording distances and different latitudes. In our experiments we demonstrate how the proposed tiling system allows to improve the results of current state-of-the-art semantic segmentation models. To foster future research we make the source code publicly available.
翻译:为应对地球观测影像语义分割计算过程中的高要求,当前最先进的流水线会将原始数据分割为更小的影像。现有方法与基准数据集常依赖基于像素的瓦片划分方案或网络地图应用采用的地理瓦片方案。子影像(包含尺寸、位置与朝向)的选择至关重要,因其既影响每个像素的可用上下文信息,又定义了训练过程中的瓦片数量,同时还会影响对瓦片内容进行下采样与上采样至分割模型所需尺寸时的信息退化程度。本文提出一种面向地球观测影像的新分割流水线,其采用基于栅格数据地理信息创建地理瓦片的划分方案。与基于像素或常规网络地图方案相比,此方法展现出多项优势:例如,所提瓦片方案在瓦片粒度、步长与影像边界对齐方面具备灵活的自定义特性,从而可在训练过程中执行瓦片定制的数据增强,并在推理阶段利用重叠瓦片数据替换上下文信息有限的像素预测。此外,生成瓦片针对异构传感器、不同拍摄距离及不同纬度均能保持一致的空间覆盖范围。实验结果表明,所提瓦片系统可有效提升当前最先进语义分割模型的性能。为促进后续研究,我们将公开本研究的源代码。