This paper introduces SCOPE-RL, a comprehensive open-source Python software designed for offline reinforcement learning (offline RL), off-policy evaluation (OPE), and selection (OPS). Unlike most existing libraries that focus solely on either policy learning or evaluation, SCOPE-RL seamlessly integrates these two key aspects, facilitating flexible and complete implementations of both offline RL and OPE processes. SCOPE-RL put particular emphasis on its OPE modules, offering a range of OPE estimators and robust evaluation-of-OPE protocols. This approach enables more in-depth and reliable OPE compared to other packages. For instance, SCOPE-RL enhances OPE by estimating the entire reward distribution under a policy rather than its mere point-wise expected value. Additionally, SCOPE-RL provides a more thorough evaluation-of-OPE by presenting the risk-return tradeoff in OPE results, extending beyond mere accuracy evaluations in existing OPE literature. SCOPE-RL is designed with user accessibility in mind. Its user-friendly APIs, comprehensive documentation, and a variety of easy-to-follow examples assist researchers and practitioners in efficiently implementing and experimenting with various offline RL methods and OPE estimators, tailored to their specific problem contexts. The documentation of SCOPE-RL is available at https://scope-rl.readthedocs.io/en/latest/.

翻译：本文介绍了SCOPE-RL——一个专为离线强化学习（offline RL）、离线策略评估（OPE）及离线策略选择（OPS）设计的综合性开源Python软件。与多数仅聚焦于策略学习或策略评估的现有库不同，SCOPE-RL无缝整合了这两个关键方面，支持离线RL与OPE流程的灵活且完整实现。SCOPE-RL特别强调其OPE模块，提供了丰富的OPE估计器及稳健的OPE评估协议。相较于其他软件包，该方法能实现更深入、更可靠的OPE。例如，通过估计策略下完整奖励分布而非仅关注点期望值，SCOPE-RL增强了OPE能力；同时通过呈现OPE结果中的风险-回报权衡（超越现有OPE文献中仅有的准确性评估），提供更全面的OPE评估。SCOPE-RL以用户易用性为设计核心，其用户友好型API、详尽文档及多样化易用示例，可帮助研究人员与实践者针对特定问题场景高效实现并实验各类离线RL方法与OPE估计器。SCOPE-RL文档详见https://scope-rl.readthedocs.io/en/latest/。