We propose a distributional framework for offline Inverse Reinforcement Learning (IRL) that jointly models uncertainty over reward functions and full distributions of returns. Unlike conventional IRL approaches that recover a deterministic reward estimate or match only expected returns, our method captures richer structure in expert behavior, particularly in learning the reward distribution, by minimizing first-order stochastic dominance (FSD) violations and thus integrating distortion risk measures (DRMs) into policy learning, enabling the recovery of both reward distributions and distribution-aware policies. This formulation is well-suited for behavior analysis and risk-aware imitation learning. Empirical results on synthetic benchmarks, real-world neurobehavioral data, and MuJoCo control tasks demonstrate that our method recovers expressive reward representations and achieves state-of-the-art imitation performance.

翻译：我们提出了一种用于离线逆强化学习（IRL）的分布式框架，该框架联合建模了奖励函数的不确定性以及回报的完整分布。与仅恢复确定性奖励估计或仅匹配期望回报的传统IRL方法不同，我们的方法通过最小化一阶随机占优（FSD）违规，从而将失真风险度量（DRM）整合到策略学习中，捕获了专家行为中更丰富的结构（特别是在学习奖励分布方面），使得我们能够同时恢复奖励分布和具备分布感知能力的策略。此公式化方法非常适用于行为分析和风险感知的模仿学习。在合成基准测试、真实世界神经行为数据以及MuJoCo控制任务上的实证结果表明，我们的方法能够恢复富有表现力的奖励表示，并实现了最先进的模仿性能。