Towards designing learned optimization algorithms that are usable beyond their training setting, we identify key principles that classical algorithms obey, but have up to now, not been used for Learning to Optimize (L2O). Following these principles, we provide a general design pipeline, taking into account data, architecture and learning strategy, and thereby enabling a synergy between classical optimization and L2O, resulting in a philosophy of Learning Optimization Algorithms. As a consequence our learned algorithms perform well far beyond problems from the training distribution. We demonstrate the success of these novel principles by designing a new learning-enhanced BFGS algorithm and provide numerical experiments evidencing its adaptation to many settings at test time.

翻译：为设计可在训练环境之外使用的学习型优化算法，我们识别出经典算法遵循但迄今尚未应用于"学习优化"（L2O）的关键原则。依据这些原则，我们提出一个综合考虑数据、架构与学习策略的通用设计流程，从而在经典优化与L2O之间建立协同机制，形成"学习优化算法"的方法论体系。基于此，我们构建的学习算法在远超训练分布范围的问题上均表现出优异性能。通过设计新型学习增强型BFGS算法并开展数值实验，我们验证了这些新原则的有效性，实验结果表明该算法在测试阶段能自适应多种场景。