Credit assignmen, disentangling each agent's contribution to a shared reward, is a critical challenge in cooperative multi-agent reinforcement learning (MARL). To be effective, credit assignment methods must preserve the environment's optimal policy. Some recent approaches attempt this by enforcing return equivalence, where the sum of distributed rewards must equal the team reward. However, their guarantees are conditional on a learned model's regression accuracy, making them unreliable in practice. We introduce Temporal-Agent Reward Redistribution (TAR$^2$), an approach that decouples credit modeling from this constraint. A neural network learns unnormalized contribution scores, while a separate, deterministic normalization step enforces return equivalence by construction. We demonstrate that this method is equivalent to a valid Potential-Based Reward Shaping (PBRS), which guarantees the optimal policy is preserved regardless of model accuracy. Empirically, on challenging SMACLite and Google Research Football (GRF) benchmarks, TAR$^2$ accelerates learning and achieves higher final performance than strong baselines. These results establish our method as an effective solution for the agent-temporal credit assignment problem.

翻译：信用分配——即解耦每个智能体对共享奖励的贡献——是合作式多智能体强化学习（MARL）中的关键挑战。为实现有效性，信用分配方法必须保持环境的最优策略。近期一些方法尝试通过强制回报等价性（即分配奖励的总和必须等于团队奖励）来实现这一目标，但其保证条件依赖于学习模型的回归精度，导致实际应用中的可靠性不足。本文提出时域-智能体奖励再分配方法（TAR$^2$），该方法将信用建模与此约束解耦：神经网络学习未归一化的贡献分数，而独立的确定性归一化步骤通过结构设计强制实现回报等价性。我们证明该方法等价于有效的基于势函数的奖励塑形（PBRS），从而保证无论模型精度如何都能保持最优策略。在具有挑战性的SMACLite和Google Research Football（GRF）基准测试中，TAR$^2$相比强基线方法显著加速学习过程并取得更高的最终性能。这些结果表明我们的方法为解决智能体-时域信用分配问题提供了有效方案。