Modern deep learning systems are increasingly deployed in situations such as personalization and federated learning where it is necessary to support i) learning on small amounts of data, and ii) communication efficient distributed training protocols. In this work we develop FiLM Transfer (FiT) which fulfills these requirements in the image classification setting. FiT uses an automatically configured Naive Bayes classifier on top of a fixed backbone that has been pretrained on large image datasets. Parameter efficient FiLM layers are used to modulate the backbone, shaping the representation for the downstream task. The network is trained via an episodic fine-tuning protocol. The approach is parameter efficient which is key for enabling few-shot learning, inexpensive model updates for personalization, and communication efficient federated learning. We experiment with FiT on a wide range of downstream datasets and show that it achieves better classification accuracy than the state-of-the-art Big Transfer (BiT) algorithm at low-shot and on the challenging VTAB-1k benchmark, with fewer than 1% of the updateable parameters. Finally, we demonstrate the parameter efficiency of FiT in distributed low-shot applications including model personalization and federated learning where model update size is an important performance metric.

翻译：在个人化和联合学习等必要情况下,越来越多地部署现代深层学习系统,例如个人化和联合学习,从而有必要支持(i)学习少量数据,以及(ii)通信高效分布式培训协议。在这项工作中,我们开发了符合图像分类设置中这些要求的FILM传输(FIT)系统。FiT在固定主干顶部上使用一个自动配置的Naive Bayes分类器,该分类器在大型图像数据集上经过预先训练。Parater 高效的FilM层用于调节主干,为下游任务塑造代表制。网络通过一个偶发微调协议接受培训。该方法具有参数效率,这是促成少发学习、个人化的廉价模型更新以及通信高效的联邦化学习的关键。我们与FiT在一系列的下游数据集上进行了实验,并表明它比在低发和具有挑战性的VTAB-1k基准的参数得到更好的分类精确度,而更新后的参数不到1%。最后,我们展示了FitTFiT的参数效率,在分布式的低位化模型中,包括个人模型学习。