We present TopoMortar, a brick wall dataset that is the first dataset specifically designed to evaluate topology-focused image segmentation methods, such as topology loss functions. TopoMortar enables to investigate in two ways whether methods incorporate prior topological knowledge. First, by eliminating challenges seen in real-world data, such as small training set, noisy labels, and out-of-distribution test-set images, that, as we show, impact the effectiveness of topology losses. Second, by allowing to assess in the same dataset topology accuracy across dataset challenges, isolating dataset-related effects from the effect of incorporating prior topological knowledge. In these two experiments, it is deliberately difficult to improve topology accuracy without actually using topology information, thus, permitting to attribute an improvement in topology accuracy to the incorporation of prior topological knowledge. To this end, TopoMortar includes three types of labels (accurate, noisy, pseudo-labels), two fixed training sets (large and small), and in-distribution and out-of-distribution test-set images. We compared eight loss functions on TopoMortar, and we found that clDice achieved the most topologically accurate segmentations, Skeleton Recall loss performed best particularly with noisy labels, and the relative advantageousness of the other loss functions depended on the experimental setting. Additionally, we show that simple methods, such as data augmentation and self-distillation, can elevate Cross entropy Dice loss to surpass most topology loss functions, and that those simple methods can enhance topology loss functions as well. clDice and Skeleton Recall loss, both skeletonization-based loss functions, were also the fastest to train, making this type of loss function a promising research direction. TopoMortar and our code can be found at https://github.com/jmlipman/TopoMortar

翻译：我们提出了TopoMortar，一个砖墙数据集，这是首个专门设计用于评估聚焦拓扑的图像分割方法（如拓扑损失函数）的数据集。TopoMortar能够通过两种方式探究方法是否融入了先验拓扑知识。首先，通过消除现实世界数据中存在的挑战，例如小训练集、噪声标签和分布外测试集图像，正如我们所展示的，这些挑战会影响拓扑损失的有效性。其次，通过允许在同一数据集中评估跨越不同数据集挑战的拓扑精度，从而将数据集相关效应与融入先验拓扑知识的效果分离开来。在这两个实验中，刻意使得在不实际使用拓扑信息的情况下难以提升拓扑精度，因此，可以将拓扑精度的提升归因于融入了先验拓扑知识。为此，TopoMortar包含三种类型的标签（精确、噪声、伪标签）、两个固定的训练集（大和小）以及分布内和分布外的测试集图像。我们在TopoMortar上比较了八种损失函数，发现clDice实现了拓扑最精确的分割，Skeleton Recall损失在噪声标签下表现尤为出色，而其他损失函数的相对优势取决于实验设置。此外，我们展示了简单方法，如数据增强和自蒸馏，可以将交叉熵Dice损失提升到超越大多数拓扑损失函数的水平，并且这些简单方法也能增强拓扑损失函数。clDice和Skeleton Recall损失，这两种基于骨架化的损失函数，也是训练速度最快的，使得这类损失函数成为一个有前景的研究方向。TopoMortar和我们的代码可在 https://github.com/jmlipman/TopoMortar 找到。