Few-shot learning (FSL) is a central problem in meta-learning, where learners must efficiently learn from few labeled examples. Within FSL, feature pre-training has recently become an increasingly popular strategy to significantly improve generalization performance. However, the contribution of pre-training is often overlooked and understudied, with limited theoretical understanding of its impact on meta-learning performance. Further, pre-training requires a consistent set of global labels shared across training tasks, which may be unavailable in practice. In this work, we address the above issues by first showing the connection between pre-training and meta-learning. We discuss why pre-training yields more robust meta-representation and connect the theoretical analysis to existing works and empirical results. Secondly, we introduce Meta Label Learning (MeLa), a novel meta-learning algorithm that learns task relations by inferring global labels across tasks. This allows us to exploit pre-training for FSL even when global labels are unavailable or ill-defined. Lastly, we introduce an augmented pre-training procedure that further improves the learned meta-representation. Empirically, MeLa outperforms existing methods across a diverse range of benchmarks, in particular under a more challenging setting where the number of training tasks is limited and labels are task-specific. We also provide extensive ablation study to highlight its key properties.

翻译：小样本学习是元学习中的核心问题，要求学习器从少量标注样本中高效学习。近年来，特征预训练策略在小样本学习中日益流行，能显著提升泛化性能。然而，预训练的贡献常被忽视且研究不足，其对元学习性能影响的理论分析尤为有限。此外，预训练需要跨训练任务共享的全局标签集合，这在实际应用中可能难以获取。本研究首先揭示预训练与元学习之间的关联，论证为何预训练能产生更鲁棒的元表征，并将理论分析与现有工作及实证结果相联系。其次，我们提出元标签学习算法，通过跨任务推理全局标签来学习任务间关系，使得在全局标签缺失或定义不明确时仍能利用预训练进行小样本学习。最后，我们引入增强型预训练流程，进一步改善所学习的元表征。实验表明，在多样化基准测试中，MeLa算法优于现有方法，尤其在训练任务数有限且标签为任务特定场景的更具挑战性设置下表现突出。我们还通过大量消融实验揭示了其关键特性。