Semantic segmentation of multichannel images is a fundamental task for many applications. Selecting an appropriate channel combination from the original multichannel image can improve the accuracy of semantic segmentation and reduce the cost of data storage, processing and future acquisition. Existing channel selection methods typically use a reasonable selection procedure to determine a desirable channel combination, and then train a semantic segmentation network using that combination. In this study, the concept of pruning from a supernet is used for the first time to integrate the selection of channel combination and the training of a semantic segmentation network. Based on this concept, a One-Shot Task-Adaptive (OSTA) channel selection method is proposed for the semantic segmentation of multichannel images. OSTA has three stages, namely the supernet training stage, the pruning stage and the fine-tuning stage. The outcomes of six groups of experiments (L7Irish3C, L7Irish2C, L8Biome3C, L8Biome2C, RIT-18 and Semantic3D) demonstrated the effectiveness and efficiency of OSTA. OSTA achieved the highest segmentation accuracies in all tests (62.49% (mIoU), 75.40% (mIoU), 68.38% (mIoU), 87.63% (mIoU), 66.53% (mA) and 70.86% (mIoU), respectively). It even exceeded the highest accuracies of exhaustive tests (61.54% (mIoU), 74.91% (mIoU), 67.94% (mIoU), 87.32% (mIoU), 65.32% (mA) and 70.27% (mIoU), respectively), where all possible channel combinations were tested. All of this can be accomplished within a predictable and relatively efficient timeframe, ranging from 101.71% to 298.1% times the time required to train the segmentation network alone. In addition, there were interesting findings that were deemed valuable for several fields.
翻译:多通道图像的语义分割是众多应用中的基础任务。从原始多通道图像中选择合适的通道组合,既能提升语义分割精度,又能降低数据存储、处理及未来采集的成本。现有通道选择方法通常采用合理的筛选流程确定最优通道组合,再基于该组合训练语义分割网络。本研究首次将超网络剪枝概念引入通道组合选择与语义分割网络训练的联合优化过程。基于该理念,提出了一种面向多通道图像语义分割的一次性任务自适应通道选择方法(OSTA)。OSTA包含超网络训练、剪枝与微调三个阶段。六组实验(L7Irish3C、L7Irish2C、L8Biome3C、L8Biome2C、RIT-18、Semantic3D)结果验证了OSTA的有效性与高效性:在所有测试中,OSTA均取得最高分割精度(分别为62.49%(mIoU)、75.40%(mIoU)、68.38%(mIoU)、87.63%(mIoU)、66.53%(mA)及70.86%(mIoU)),甚至超越了穷举测试(测试所有可能通道组合)的最高精度(分别为61.54%(mIoU)、74.91%(mIoU)、67.94%(mIoU)、87.32%(mIoU)、65.32%(mA)及70.27%(mIoU))。整个过程可在可预测且相对高效的时间范围内完成,耗时仅为单独训练分割网络所需时间的101.71%至298.1%。此外,实验中发现若干对多个领域具有重要价值的现象。