Model-agnostic meta-learning (MAML) is one of the most successful meta-learning algorithms. It has a bi-level optimization structure where the outer-loop process learns a shared initialization and the inner-loop process optimizes task-specific weights. Although MAML relies on the standard gradient descent in the inner-loop, recent studies have shown that controlling the inner-loop's gradient descent with a meta-learned preconditioner can be beneficial. Existing preconditioners, however, cannot simultaneously adapt in a task-specific and path-dependent way. Additionally, they do not satisfy the Riemannian metric condition, which can enable the steepest descent learning with preconditioned gradient. In this study, we propose Geometry-Adaptive Preconditioned gradient descent (GAP) that can overcome the limitations in MAML; GAP can efficiently meta-learn a preconditioner that is dependent on task-specific parameters, and its preconditioner can be shown to be a Riemannian metric. Thanks to the two properties, the geometry-adaptive preconditioner is effective for improving the inner-loop optimization. Experiment results show that GAP outperforms the state-of-the-art MAML family and preconditioned gradient descent-MAML (PGD-MAML) family in a variety of few-shot learning tasks. Code is available at: https://github.com/Suhyun777/CVPR23-GAP.

翻译：模型无关元学习（MAML）是最成功的元学习算法之一。它采用双层优化结构，其中外层循环学习共享初始化参数，内层循环优化任务特定权重。尽管MAML在内层循环中依赖标准梯度下降，但近期研究表明，使用元学习预条件器控制内层梯度下降可带来益处。然而，现有预条件器无法同时以任务特定和路径依赖的方式进行自适应，且不满足黎曼度量条件——该条件能实现带预条件梯度的最速下降学习。本研究提出几何自适应预条件梯度下降（GAP）方法，可克服MAML的局限性：GAP能高效元学习依赖于任务特定参数的预条件器，且其预条件器可被证明是黎曼度量。得益于这两个特性，几何自适应预条件器能有效改善内层循环优化。实验结果表明，在各种少样本学习任务中，GAP的性能优于最先进的MAML系列及预条件梯度下降MAML（PGD-MAML）系列方法。代码开源地址：https://github.com/Suhyun777/CVPR23-GAP。