Transfer learning leverages feature representations of deep neural networks (DNNs) pretrained on source tasks with rich data to empower effective finetuning on downstream tasks. However, the pretrained models are often prohibitively large for delivering generalizable representations, which limits their deployment on edge devices with constrained resources. To close this gap, we propose a new transfer learning pipeline, which leverages our finding that robust tickets can transfer better, i.e., subnetworks drawn with properly induced adversarial robustness can win better transferability over vanilla lottery ticket subnetworks. Extensive experiments and ablation studies validate that our proposed transfer learning pipeline can achieve enhanced accuracy-sparsity trade-offs across both diverse downstream tasks and sparsity patterns, further enriching the lottery ticket hypothesis.

翻译：迁移学习利用在数据丰富的源任务上预训练的深度神经网络（DNNs）的特征表示，以促进下游任务的高效微调。然而，预训练模型通常过于庞大，难以提供泛化性强的表示，这限制了其在资源受限的边缘设备上的部署。为弥补这一不足，我们提出了一种新的迁移学习流程，该流程基于我们的发现：健壮票证能够更优地迁移，即通过适当诱导的对抗鲁棒性提取的子网络，在迁移性上优于传统彩票票证子网络。大量实验和消融研究验证了，我们提出的迁移学习流程能够在多样化的下游任务和稀疏模式中实现精度与稀疏性的更优平衡，进一步丰富了彩票票证假说。