Data efficiency, or the ability to generalize from a few labeled data, remains a major challenge in deep learning. Semi-supervised learning has thrived in traditional recognition tasks alleviating the need for large amounts of labeled data, yet it remains understudied in image-to-image translation (I2I) tasks. In this work, we introduce the first semi-supervised (semi-paired) framework for label-to-image translation, a challenging subtask of I2I which generates photorealistic images from semantic label maps. In the semi-paired setting, the model has access to a small set of paired data and a larger set of unpaired images and labels. Instead of using geometrical transformations as a pretext task like previous works, we leverage an input reconstruction task by exploiting the conditional discriminator on the paired data as a reverse generator. We propose a training algorithm for this shared network, and we present a rare classes sampling algorithm to focus on under-represented classes. Experiments on 3 standard benchmarks show that the proposed model outperforms state-of-the-art unsupervised and semi-supervised approaches, as well as some fully supervised approaches while using a much smaller number of paired samples.

翻译：数据效率（即从少量标注数据中泛化的能力）仍是深度学习的主要挑战。半监督学习已在传统识别任务中蓬勃发展，减少了对大量标注数据的需求，但在图像到图像翻译（I2I）任务中仍缺乏深入研究。本文首次提出面向标签到图像翻译的半监督（半配对）框架，该任务是I2I中具有挑战性的子任务，旨在从语义标签图生成逼真图像。在半配对设置中，模型可获取少量配对数据及大量未配对图像与标签。不同于先前工作将几何变换作为预文本任务，我们通过利用配对数据上的条件鉴别器作为逆向生成器，构建输入重建任务。我们提出了该共享网络的训练算法，并设计稀有类别采样算法以聚焦于欠表征类别。在3个标准基准上的实验表明，所提模型在仅使用极少配对样本的情况下，性能优于当前最先进的非监督、半监督方法及部分全监督方法。