Conventional optimization methods in machine learning and controls rely heavily on first-order update rules. Selecting the right method and hyperparameters for a particular task often involves trial-and-error or practitioner intuition, motivating the field of meta-learning. We generalize a broad family of preexisting update rules by proposing a meta-learning framework in which the inner loop optimization step involves solving a differentiable convex optimization (DCO). We illustrate the theoretical appeal of this approach by showing that it enables one-step optimization of a family of linear least squares problems, given that the meta-learner has sufficient exposure to similar tasks. Various instantiations of the DCO update rule are compared to conventional optimizers on a range of illustrative experimental settings.

翻译：人工智能与控制领域的传统优化方法高度依赖一阶更新规则。针对特定任务选择合适的方法与超参数，往往需要反复试错或依赖实践者的直觉，这推动了元学习领域的发展。我们通过提出一种元学习框架，将内层循环优化步骤转化为求解可微凸优化问题，从而推广了现有的大量更新规则。理论分析表明，当元学习器充分接触相似任务时，该方法能够实现线性最小二乘问题族的一步优化。通过一系列实验设置，我们将可微凸优化更新规则的各种实例化结果与常规优化器进行了对比。