Surrogate-assisted Evolutionary Algorithms~(SAEAs) have shown promising robustness in solving expensive optimization problems. A key aspect that impacts SAEAs' effectiveness is surrogate model selection, which in existing works is predominantly decided by human developer. Such human-made design choice introduces strong bias into SAEAs and may hurt their expected performance on out-of-scope tasks. In this paper, we propose a reinforcement learning-assisted ensemble framework, termed as SEEMOO, which is capable of scheduling different surrogate models within a single optimization process, hence boosting the overall optimization performance in a cooperative paradigm. Specifically, we focus on expensive multi-objective optimization problems, where multiple objective functions shape a compositional landscape and hence challenge surrogate selection. SEEMOO comprises following core designs: 1) A pre-collected model pool that maintains different surrogate models; 2) An attention-based state-extractor supports universal optimization state representation of problems with varied objective numbers; 3) a deep Q-network serves as dynamic surrogate selector: Given the optimization state, it selects desired surrogate model for current-step evaluation. SEEMOO is trained to maximize the overall optimization performance under a training problem distribution. Extensive benchmark results demonstrate SEEMOO's surrogate ensemble paradigm boosts the optimization performance of single-surrogate baselines. Further ablation studies underscore the importance of SEEMOO's design components.

翻译：代理辅助进化算法在求解昂贵优化问题中展现出良好的鲁棒性。影响算法效能的关键因素是代理模型选择，现有研究主要依赖开发者人工决策。这种人为设计选择会给算法引入强烈偏差，并可能损害其在超范围任务上的预期性能。本文提出一种强化学习辅助的集成框架SEEMOO，能够在单一优化过程中调度不同的代理模型，从而以协同范式提升整体优化性能。具体而言，我们聚焦于昂贵多目标优化问题，其中多个目标函数构成复合型解空间形态，从而对代理选择形成挑战。SEEMOO包含以下核心设计：1）预构建的模型池维护多种代理模型；2）基于注意力机制的状态提取器支持不同目标数问题的通用优化状态表征；3）深度Q网络作为动态代理选择器：根据优化状态为当前步评估选择所需代理模型。SEEMOO通过训练问题分布进行训练以最大化整体优化性能。大量基准测试结果表明SEEMOO的代理集成范式显著提升了单代理基线的优化性能。进一步的消融研究验证了SEEMOO各设计组件的重要性。