The goal of data-free meta-learning is to learn useful prior knowledge from a collection of pre-trained models without accessing their training data. However, existing works only solve the problem in parameter space, which (i) ignore the fruitful data knowledge contained in the pre-trained models; (ii) can not scale to large-scale pre-trained models; (iii) can only meta-learn pre-trained models with the same network architecture. To address those issues, we propose a unified framework, dubbed PURER, which contains: (1) ePisode cUrriculum inveRsion (ECI) during data-free meta training; and (2) invErsion calibRation following inner loop (ICFIL) during meta testing. During meta training, we propose ECI to perform pseudo episode training for learning to adapt fast to new unseen tasks. Specifically, we progressively synthesize a sequence of pseudo episodes by distilling the training data from each pre-trained model. The ECI adaptively increases the difficulty level of pseudo episodes according to the real-time feedback of the meta model. We formulate the optimization process of meta training with ECI as an adversarial form in an end-to-end manner. During meta testing, we further propose a simple plug-and-play supplement-ICFIL-only used during meta testing to narrow the gap between meta training and meta testing task distribution. Extensive experiments in various real-world scenarios show the superior performance of ours.

翻译：无数据元学习的目标是从一组预训练模型中学习有用的先验知识，而无需访问其训练数据。然而，现有方法仅在参数空间中解决该问题，这存在以下不足：（i）忽略了预训练模型蕴含的丰富数据知识；（ii）无法扩展到大规模预训练模型；（iii）仅能对具有相同网络架构的预训练模型进行元学习。针对上述问题，我们提出一个统一框架PURER，包含两个核心模块：（1）无数据元训练阶段的情节课程反演（ECI）；（2）元测试阶段的内循环后反演校准（ICFIL）。在元训练阶段，我们提出ECI方法，通过构建伪情节训练来学习快速适应新未见任务的能力。具体而言，我们从每个预训练模型中蒸馏训练数据，逐步合成一系列伪情节。ECI根据元模型的实时反馈自适应提升伪情节的难度级别。我们将包含ECI的元训练优化过程以端到端方式建模为对抗形式。在元测试阶段，我们进一步提出一个简单的即插即用补充模块——仅在元测试时使用的ICFIL，以缩小元训练与元测试任务分布之间的差距。在多种真实场景下的大量实验表明，我们的方法具有优越性能。