In this paper, we hypothesize that gradient-based meta-learning (GBML) implicitly suppresses the Hessian along the optimization trajectory in the inner loop. Based on this hypothesis, we introduce an algorithm called SHOT (Suppressing the Hessian along the Optimization Trajectory) that minimizes the distance between the parameters of the target and reference models to suppress the Hessian in the inner loop. Despite dealing with high-order terms, SHOT does not increase the computational complexity of the baseline model much. It is agnostic to both the algorithm and architecture used in GBML, making it highly versatile and applicable to any GBML baseline. To validate the effectiveness of SHOT, we conduct empirical tests on standard few-shot learning tasks and qualitatively analyze its dynamics. We confirm our hypothesis empirically and demonstrate that SHOT outperforms the corresponding baseline. Code is available at: https://github.com/JunHoo-Lee/SHOT

翻译：[翻译摘要] 本文提出假设：梯度元学习（GBML）在内循环中沿优化轨迹隐式抑制Hessian矩阵。基于该假设，我们引入名为SHOT（沿优化轨迹抑制Hessian矩阵）的算法，通过最小化目标与参考模型的参数距离来抑制内循环中的Hessian矩阵。尽管涉及高阶项，SHOT并未显著增加基线模型的计算复杂度。该算法与GBML中使用的算法和架构无关，具有高度通用性，可应用于任意GBML基线。为验证SHOT的有效性，我们在标准小样本学习任务上进行实证测试，并定性分析其动力学特性。实证结果证实了我们的假设，并表明SHOT的性能优于相应基线。代码开源地址：https://github.com/JunHoo-Lee/SHOT