This work investigates the unexplored usability of self-supervised representation learning in the direction of functional knowledge transfer. In this work, functional knowledge transfer is achieved by joint optimization of self-supervised learning pseudo task and supervised learning task, improving supervised learning task performance. Recent progress in self-supervised learning uses a large volume of data, which becomes a constraint for its applications on small-scale datasets. This work shares a simple yet effective joint training framework that reinforces human-supervised task learning by learning self-supervised representations just-in-time and vice versa. Experiments on three public datasets from different visual domains, Intel Image, CIFAR, and APTOS, reveal a consistent track of performance improvements on classification tasks during joint optimization. Qualitative analysis also supports the robustness of learnt representations. Source code and trained models are available on GitHub.

翻译：本研究探讨了自监督表示学习在功能性知识迁移方向上尚未探索的实用性。通过联合优化自监督学习伪任务与监督学习任务，实现了功能性知识迁移，从而提升了监督学习任务的性能。近期自监督学习的进展依赖于大规模数据，这限制了其在小型数据集上的应用。本文提出了一种简单而有效的联合训练框架，通过即时学习自监督表示来增强人类监督任务学习，反之亦然。在来自不同视觉领域的三个公开数据集（Intel Image、CIFAR 和 APTOS）上的实验表明，联合优化能在分类任务上持续提升性能。定性分析也验证了所学表示的鲁棒性。源代码和训练模型已发布于 GitHub。