Data-Free Meta-Learning (DFML) aims to extract knowledge from a collection of pre-trained models without requiring the original data, presenting practical benefits in contexts constrained by data privacy concerns. Current DFML methods primarily focus on the data recovery from these pre-trained models. However, they suffer from slow recovery speed and overlook gaps inherent in heterogeneous pre-trained models. In response to these challenges, we introduce the Faster and Better Data-Free Meta-Learning (FREE) framework, which contains: (i) a meta-generator for rapidly recovering training tasks from pre-trained models; and (ii) a meta-learner for generalizing to new unseen tasks. Specifically, within the module Faster Inversion via Meta-Generator, each pre-trained model is perceived as a distinct task. The meta-generator can rapidly adapt to a specific task in just five steps, significantly accelerating the data recovery. Furthermore, we propose Better Generalization via Meta-Learner and introduce an implicit gradient alignment algorithm to optimize the meta-learner. This is achieved as aligned gradient directions alleviate potential conflicts among tasks from heterogeneous pre-trained models. Empirical experiments on multiple benchmarks affirm the superiority of our approach, marking a notable speed-up (20$\times$) and performance enhancement (1.42\% $\sim$ 4.78\%) in comparison to the state-of-the-art.

翻译：无数据元学习（Data-Free Meta-Learning, DFML）旨在从预训练模型集合中提取知识，无需原始数据，在受数据隐私问题约束的背景下具有实际优势。当前的DFML方法主要关注从这些预训练模型中恢复数据。然而，它们面临恢复速度慢且忽视异构预训练模型固有差距的问题。针对这些挑战，我们提出了更快更好的无数据元学习（FREE）框架，包含：(i) 用于从预训练模型中快速恢复训练任务的元生成器；(ii) 用于泛化至新未见任务的元学习器。具体而言，在“通过元生成器实现更快反演”模块中，每个预训练模型被视为一个独立任务。元生成器仅需五步即可快速适应特定任务，显著加速数据恢复。此外，我们提出“通过元学习器实现更好泛化”，并引入隐式梯度对齐算法以优化元学习器。这通过对齐梯度方向来缓解异构预训练模型间的潜在任务冲突。在多个基准上的实证实验证实了我们方法的优越性，相比现有最优方法实现了显著的加速（20倍）和性能提升（1.42%~4.78%）。