Self-training allows a network to learn from the predictions of a more complicated model, thus often requires well-trained teacher models and mixture of teacher-student data while multi-task learning jointly optimizes different targets to learn salient interrelationship and requires multi-task annotations for each training example. These frameworks, despite being particularly data demanding have potentials for data exploitation if such assumptions can be relaxed. In this paper, we compare self-training object detection under the deficiency of teacher training data where students are trained on unseen examples by the teacher, and multi-task learning with partially annotated data, i.e. single-task annotation per training example. Both scenarios have their own limitation but potentially helpful with limited annotated data. Experimental results show the improvement of performance when using a weak teacher with unseen data for training a multi-task student. Despite the limited setup we believe the experimental results show the potential of multi-task knowledge distillation and self-training, which could be beneficial for future study. Source code is at https://lhoangan.github.io/multas.

翻译：自训练允许网络从更复杂模型的预测中学习，因此通常需要训练有素的教师模型以及教师-学生数据的混合，而多任务学习则通过联合优化不同目标来学习显著的相关性，并需要为每个训练样本提供多任务标注。尽管这些框架对数据需求较高，但如果能放宽这些假设，它们具有数据利用的潜力。在本文中，我们比较了教师训练数据不足情况下的自训练目标检测（其中学生由教师对未见过的样本进行训练）以及部分标注数据（即每个训练样本仅含单任务标注）下的多任务学习。两种场景各有其局限性，但在有限标注数据的情况下可能具有帮助。实验结果表明，当使用弱教师结合未见数据训练多任务学生时，性能得到了提升。尽管设置有限，但我们认为实验结果展示了多任务知识蒸馏与自训练的潜力，这可能对未来的研究有所裨益。源代码见 https://lhoangan.github.io/multas。