Neutronic calculations for reactors are a daunting task when using Monte Carlo (MC) methods. As high-performance computing has advanced, the simulation of a reactor is nowadays more readily done, but design and optimization with multiple parameters is still a computational challenge. MC transport simulations, coupled with machine learning techniques, offer promising avenues for enhancing the efficiency and effectiveness of nuclear reactor optimization. This paper introduces a novel benchmark problem within the OpenNeoMC framework designed specifically for reinforcement learning. The benchmark involves optimizing a unit cell of a research reactor with two varying parameters (fuel density and water spacing) to maximize neutron flux while maintaining reactor criticality. The test case features distinct local optima, representing different physical regimes, thus posing a challenge for learning algorithms. Through extensive simulations utilizing evolutionary and neuroevolutionary algorithms, we demonstrate the effectiveness of reinforcement learning in navigating complex optimization landscapes with strict constraints. Furthermore, we propose acceleration techniques within the OpenNeoMC framework, including model updating and cross-section usage by RAM utilization, to expedite simulation times. Our findings emphasize the importance of machine learning integration in reactor optimization and contribute to advancing methodologies for addressing intricate optimization challenges in nuclear engineering. The sources of this work are available at our GitHub repository: https://github.com/Scientific-Computing-Lab-NRCN/RLOpenNeoMC

翻译：采用蒙特卡罗方法进行反应堆中子学计算是一项艰巨任务。随着高性能计算的发展，反应堆模拟如今已更容易实现，但涉及多参数的设计与优化仍面临计算挑战。蒙特卡罗输运模拟与机器学习技术相结合，为提升核反应堆优化的效率与效果提供了可行途径。本文在OpenNeoMC框架内引入一个专为强化学习设计的新型基准测试问题。该基准通过优化研究反应堆单胞的两个可变参数（燃料密度与水层间距），在保持反应堆临界性的同时最大化中子通量。该测试用例包含代表不同物理机制的独特局部最优解，从而对学习算法构成挑战。通过采用进化算法与神经进化算法的广泛模拟，我们证明了强化学习在严格约束下驾驭复杂优化空间的有效性。此外，我们在OpenNeoMC框架内提出加速技术，包括通过RAM利用率实现模型更新与截面使用，以缩短模拟时间。我们的研究结果强调了机器学习在反应堆优化中集成的重要性，并为解决核工程中复杂优化难题的方法学发展做出贡献。本研究源码已发布于GitHub仓库：https://github.com/Scientific-Computing-Lab-NRCN/RLOpenNeoMC