The surge in reinforcement learning (RL) applications gave rise to diverse supporting technology, such as RL frameworks. However, the architectural patterns of these frameworks are inconsistent across implementations and there exists no reference architecture (RA) to form a common basis of comparison, evaluation, and integration. To address this gap, we propose an RA of RL frameworks. Through a grounded theory approach, we analyze 18 state-of-the-practice RL frameworks and, by that, we identify recurring architectural components and their relationships, and codify them in an RA. To demonstrate our RA, we reconstruct characteristic RL patterns. Finally, we identify architectural trends, e.g., commonly used components, and outline paths to improving RL frameworks.

翻译：强化学习（RL）应用的激增催生了多样化的支持技术，例如RL框架。然而，这些框架的架构模式在不同实现中并不一致，且缺乏一个参考架构（RA）来形成比较、评估和集成的共同基础。为填补这一空白，我们提出了一个RL框架的RA。通过扎根理论方法，我们分析了18个当前实践中的先进RL框架，并以此识别出反复出现的架构组件及其关系，将其编码为一个RA。为展示我们的RA，我们重构了具有代表性的RL模式。最后，我们识别了架构趋势，例如常用组件，并概述了改进RL框架的路径。