Recent Meta-learning for Black-Box Optimization (MetaBBO) methods harness neural networks to meta-learn configurations of traditional black-box optimizers. Despite their success, they are inevitably restricted by the limitations of predefined hand-crafted optimizers. In this paper, we present \textsc{Symbol}, a novel framework that promotes the automated discovery of black-box optimizers through symbolic equation learning. Specifically, we propose a Symbolic Equation Generator (SEG) that allows closed-form optimization rules to be dynamically generated for specific tasks and optimization steps. Within \textsc{Symbol}, we then develop three distinct strategies based on reinforcement learning, so as to meta-learn the SEG efficiently. Extensive experiments reveal that the optimizers generated by \textsc{Symbol} not only surpass the state-of-the-art BBO and MetaBBO baselines, but also exhibit exceptional zero-shot generalization abilities across entirely unseen tasks with different problem dimensions, population sizes, and optimization horizons. Furthermore, we conduct in-depth analyses of our \textsc{Symbol} framework and the optimization rules that it generates, underscoring its desirable flexibility and interpretability.
翻译:摘要:近期用于黑箱优化的元学习方法(MetaBBO)利用神经网络元学习传统黑箱优化器的配置。尽管这些方法取得了成功,但它们不可避免地受到预定义手工优化器局限性的束缚。本文提出\textsc{Symbol}这一新颖框架,通过符号方程学习促进黑箱优化器的自动发现。具体而言,我们提出符号方程生成器(SEG),可针对特定任务和优化步骤动态生成闭式优化规则。在\textsc{Symbol}框架内,我们进一步基于强化学习开发了三种不同策略,以实现对SEG的高效元学习。广泛实验表明,由\textsc{Symbol}生成的优化器不仅超越了最先进的BBO和MetaBBO基线,还在完全未见任务(具有不同问题维度、种群规模和优化周期)中展现出卓越的零样本泛化能力。此外,我们对\textsc{Symbol}框架及其生成的优化规则进行了深入分析,彰显了其理想的灵活性与可解释性。