Learning control policies offline from pre-recorded datasets is a promising avenue for solving challenging real-world problems. However, available datasets are typically of mixed quality, with a limited number of the trajectories that we would consider as positive examples; i.e., high-quality demonstrations. Therefore, we propose a novel iterative learning algorithm for identifying expert trajectories in unlabeled mixed-quality robotics datasets given a minimal set of positive examples, surpassing existing algorithms in terms of accuracy. We show that applying behavioral cloning to the resulting filtered dataset outperforms several competitive offline reinforcement learning and imitation learning baselines. We perform experiments on a range of simulated locomotion tasks and on two challenging manipulation tasks on a real robotic system; in these experiments, our method showcases state-of-the-art performance. Our website: \url{https://sites.google.com/view/offline-policy-learning-pubc}.

翻译：摘要：从预记录数据集中离线学习控制策略为解决现实世界的复杂问题提供了一条有前景的途径。然而，可用数据集通常质量参差不齐，其中被视为正例（即高质量示范）的轨迹数量有限。为此，我们提出了一种新颖的迭代学习算法，能够在给定最小正例集的情况下，从无标记的混合质量机器人数据集中识别专家轨迹，在准确性上超越了现有算法。研究表明，对所得过滤数据集应用行为克隆，其性能优于多种具有竞争力的离线强化学习和模仿学习基线方法。我们在模拟运动任务及真实机器人系统上的两项具有挑战性的操作任务中开展了实验；实验结果表明，我们的方法达到了最先进的性能。我们的网站：\url{https://sites.google.com/view/offline-policy-learning-pubc}。