A novel method, the Pareto Envelope Augmented with Reinforcement Learning (PEARL), has been developed to address the challenges posed by multi-objective problems, particularly in the field of engineering where the evaluation of candidate solutions can be time-consuming. PEARL distinguishes itself from traditional policy-based multi-objective Reinforcement Learning methods by learning a single policy, eliminating the need for multiple neural networks to independently solve simpler sub-problems. Several versions inspired from deep learning and evolutionary techniques have been crafted, catering to both unconstrained and constrained problem domains. Curriculum Learning is harnessed to effectively manage constraints in these versions. PEARL's performance is first evaluated on classical multi-objective benchmarks. Additionally, it is tested on two practical PWR core Loading Pattern optimization problems to showcase its real-world applicability. The first problem involves optimizing the Cycle length and the rod-integrated peaking factor as the primary objectives, while the second problem incorporates the mean average enrichment as an additional objective. Furthermore, PEARL addresses three types of constraints related to boron concentration, peak pin burnup, and peak pin power. The results are systematically compared against a conventional approach, the Non-dominated Sorting Genetic Algorithm. Notably, PEARL, specifically the PEARL-NdS variant, efficiently uncovers a Pareto front without necessitating additional efforts from the algorithm designer, as opposed to a single optimization with scaled objectives. It also outperforms the classical approach across multiple performance metrics, including the Hyper-volume.

翻译：本文提出了一种新型方法——基于强化学习的帕累托包络增强方法（PEARL），旨在解决多目标问题带来的挑战，特别是在工程领域中候选解评估耗时的问题。PEARL与传统基于策略的多目标强化学习方法不同，通过学习单一策略，无需依赖多个神经网络独立求解简单的子问题。受深度学习与进化技术启发，我们设计了多个版本，分别适用于无约束和约束问题域。通过课程学习有效管理约束条件。PEARL首先在经典多目标基准测试中评估性能，并在两个实际压水堆堆芯装载模式优化问题中验证其实际应用价值。第一个问题以循环长度和棒积分峰值因子为主要优化目标，第二个问题则引入平均富集度作为额外目标。此外，PEARL处理了与硼浓度、峰值燃料棒燃耗和峰值燃料棒功率相关的三类约束。系统性地将结果与传统非支配排序遗传算法进行对比。值得注意的是，PEARL（特别是PEARL-NdS变体）无需算法设计者付出额外努力即可高效发现帕累托前沿，这不同于采用缩放目标的单次优化方法。该方法在包含超体积在内的多个性能指标上均优于传统方法。