We propose a framework for the automatic one-shot segmentation of synthetic images generated by a StyleGAN. Our framework is based on the observation that the multi-scale hidden features in the GAN generator hold useful semantic information that can be utilized for automatic on-the-fly segmentation of the generated images. Using these features, our framework learns to segment synthetic images using a self-supervised contrastive clustering algorithm that projects the hidden features into a compact space for per-pixel classification. This contrastive learner is based on using a novel data augmentation strategy and a pixel-wise swapped prediction loss that leads to faster learning of the feature vectors for one-shot segmentation. We have tested our implementation on five standard benchmarks to yield a segmentation performance that not only outperforms the semi-supervised baselines by an average wIoU margin of 1.02 % but also improves the inference speeds by a factor of 4.5. Finally, we also show the results of using the proposed one-shot learner in implementing BagGAN, a framework for producing annotated synthetic baggage X-ray scans for threat detection. This framework was trained and tested on the PIDRay baggage benchmark to yield a performance comparable to its baseline segmenter based on manual annotations.

翻译：我们提出了一种框架，用于自动实现StyleGAN生成合成图像的单样本分割。该框架基于以下观察：GAN生成器中的多尺度隐藏特征包含有用的语义信息，可自动用于对生成图像进行即时分割。利用这些特征，我们的框架通过自监督对比聚类算法学习分割合成图像，该算法将隐藏特征投影到紧凑空间以进行逐像素分类。这一对比学习器基于新型数据增强策略和像素级交换预测损失函数，可加速单样本分割中特征向量的学习。我们在五个标准基准上测试了实现方案，分割性能不仅以平均加权交并比1.02%的优势超越半监督基线方法，还将推理速度提升4.5倍。最后，我们展示了将该单样本学习器应用于BagGAN框架的成果——该框架用于生成带注释的合成行李X光扫描图像以检测威胁。该框架在PIDRay行李基准上训练和测试，其性能与基于手动注释的基线分割器相当。