In data-rich domains such as vision, language, and speech, deep learning prevails to deliver high-performance task-specific models and can even learn general task-agnostic representations for efficient finetuning to downstream tasks. However, deep learning in resource-limited domains still faces multiple challenges including (i) limited data, (ii) constrained model development cost, and (iii) lack of adequate pre-trained models for effective finetuning. This paper provides an overview of model reprogramming to bridge this gap. Model reprogramming enables resource-efficient cross-domain machine learning by repurposing and reusing a well-developed pre-trained model from a source domain to solve tasks in a target domain without model finetuning, where the source and target domains can be vastly different. In many applications, model reprogramming outperforms transfer learning and training from scratch. This paper elucidates the methodology of model reprogramming, summarizes existing use cases, provides a theoretical explanation of the success of model reprogramming, and concludes with a discussion on open-ended research questions and opportunities. A list of model reprogramming studies is actively maintained and updated at https://github.com/IBM/model-reprogramming.

翻译：在数据丰富的领域（如视觉、语言和语音）中，深度学习通过构建高性能的特定任务模型占据主导地位，甚至能学习通用的任务无关表示以高效微调至下游任务。然而，在资源受限的领域，深度学习仍面临多重挑战，包括：（i）数据有限；（ii）模型开发成本受限；（iii）缺乏适用于微调的有效预训练模型。本文综述了模型重编程（Model Reprogramming）以弥合这一差距。模型重编程通过重用和重新利用源域中成熟的预训练模型来求解目标域任务，无需对模型进行微调，且源域与目标域可能差异显著，从而实现了资源高效的跨域机器学习。在许多应用中，模型重编程的表现优于迁移学习和从头训练。本文阐明了模型重编程的方法论，总结了现有应用案例，提供了其成功实现的理论解释，并最后讨论了开放性的研究问题与机遇。相关模型重编程的研究列表已持续维护更新于 https://github.com/IBM/model-reprogramming。