Optimization refers to the task of finding extrema of an objective function. Classical gradient-based optimizers are highly sensitive to hyperparameter choices. In highly non-convex settings their performance relies on carefully tuned learning rates, momentum, and gradient accumulation. To address these limitations, we introduce POP (Prior-fitted Optimizer Policies), a meta-learned optimizer that predicts coordinate-wise step sizes conditioned on the contextual information provided in the optimization trajectory. Our model is learned on millions of synthetic optimization problems sampled from a novel prior spanning both convex and non-convex objectives. We evaluate POP on an established benchmark including 47 optimization functions of various complexity, where it consistently outperforms first-order gradient-based methods, non-convex optimization approaches (e.g., evolutionary strategies), Bayesian optimization, and a recent meta-learned competitor under matched budget constraints. Our evaluation demonstrates strong generalization capabilities without task-specific tuning.

翻译：优化是指寻找目标函数极值的任务。经典的基于梯度的优化器对超参数选择高度敏感。在高度非凸的场景中，其性能依赖于精心调整的学习率、动量和梯度累积。为应对这些局限性，我们提出POP（先验拟合优化器策略），这是一种元学习优化器，能够根据优化轨迹提供的上下文信息预测逐坐标步长。我们的模型通过从涵盖凸与非凸目标函数的新颖先验分布中采样数百万个合成优化问题进行训练。我们在包含47个不同复杂度优化函数的基准测试集上评估POP，结果表明在相同预算约束下，POP始终优于一阶梯度方法、非凸优化方法（如进化策略）、贝叶斯优化以及近期提出的元学习竞争对手。我们的评估证明了该方法无需任务特定调优即具备强大的泛化能力。