Despite significant advances in Reinforcement Learning (RL), model performance remains highly sensitive to algorithm and hyperparameter configurations, while generalization gaps across environments complicate real-world deployment. Although prior work has studied RL generalization, the relative contribution of specific configurations to the generalization gap has not been quantitatively decomposed and systematically leveraged for configuration selection. To address this limitation, we propose an explainable framework that evaluates RL performance across robotic environments using SHapley Additive exPlanations (SHAP) to quantify configuration impacts. We establish a theoretical foundation connecting Shapley values to generalizability, empirically analyze configuration impact patterns, and introduce SHAP-guided configuration selection to enhance generalization. Our results reveal distinct patterns across algorithms and hyperparameters, with consistent configuration impacts across diverse tasks and environments. By applying these insights to configuration selection, we achieve improved RL generalizability and provide actionable guidance for practitioners.

翻译：尽管强化学习（Reinforcement Learning, RL）已取得显著进展，模型性能仍对算法与超参数配置高度敏感，而跨环境的泛化差距进一步阻碍了实际部署。尽管已有研究探讨了RL的泛化问题，但尚未对特定配置在泛化差距中的相对贡献进行量化分解，并将其系统性地用于配置选择。为解决这一局限，我们提出一个可解释框架，利用沙普利加法解释（SHapley Additive exPlanations, SHAP）量化配置影响，评估RL在机器人环境中的表现。我们建立了将沙普利值与泛化性关联的理论基础，实证分析了配置影响模式，并引入了SHAP引导的配置选择方法来增强泛化能力。我们的结果揭示了算法与超参数的不同影响模式，且在多样化任务与环境中配置影响具有一致性。通过将这些洞察应用于配置选择，我们提升了RL的泛化性，并为实践者提供了可操作指导。