Modern deep learning systems are increasingly deployed in situations such as personalization and federated learning where it is necessary to support i) learning on small amounts of data, and ii) communication efficient distributed training protocols. In this work, we develop FiLM Transfer (FiT) which fulfills these requirements in the image classification setting by combining ideas from transfer learning (fixed pretrained backbones and fine-tuned FiLM adapter layers) and meta-learning (automatically configured Naive Bayes classifiers and episodic training) to yield parameter efficient models with superior classification accuracy at low-shot. The resulting parameter efficiency is key for enabling few-shot learning, inexpensive model updates for personalization, and communication efficient federated learning. We experiment with FiT on a wide range of downstream datasets and show that it achieves better classification accuracy than the leading Big Transfer (BiT) algorithm at low-shot and achieves state-of-the art accuracy on the challenging VTAB-1k benchmark, with fewer than 1% of the updateable parameters. Finally, we demonstrate the parameter efficiency and superior accuracy of FiT in distributed low-shot applications including model personalization and federated learning where model update size is an important performance metric.

翻译：现代深度学习系统越来越多地部署在个性化和联邦学习等场景中，这些场景需要支持：i）基于少量数据的学习，以及ii）通信高效的分布式训练协议。本文开发了FiLM迁移（FiT）方法，该方法通过结合迁移学习（固定预训练骨干网络和微调FiLM适配层）和元学习（自动配置的朴素贝叶斯分类器和情景训练）的思想，生成了在低样本条件下具有卓越分类精度的参数高效模型，从而满足图像分类场景中的上述需求。由此产生的参数高效性对于实现少样本学习、低成本的个性化模型更新以及通信高效的联邦学习至关重要。我们在广泛的下游数据集上对FiT进行了实验，结果表明，在低样本条件下，它比领先的Big Transfer（BiT）算法实现了更高的分类精度，并且在具有挑战性的VTAB-1k基准测试中以少于1%的可更新参数取得了最先进的精度。最后，我们展示了FiT在分布式低样本应用（包括模型个性化和联邦学习，其中模型更新大小是重要的性能指标）中的参数高效性和优越精度。