In an unfamiliar setting, a model-based reinforcement learning agent can be limited by the accuracy of its world model. In this work, we present a novel, training-free approach to improving the performance of such agents separately from planning and learning. We do so by applying iterative inference at decision-time, to fine-tune the inferred agent states based on the coherence of future state representations. Our approach achieves a consistent improvement in both reconstruction accuracy and task performance when applied to visual 3D navigation tasks. We go on to show that considering more future states further improves the performance of the agent in partially-observable environments, but not in a fully-observable one. Finally, we demonstrate that agents with less training pre-evaluation benefit most from our approach.

翻译：在陌生环境中，基于模型的强化学习智能体可能受限于其世界模型的准确性。本文提出了一种无需训练的新方法，用于在规划与学习之外独立提升此类智能体的性能。该方法通过在决策时应用迭代推理，根据未来状态表征的一致性对推断出的智能体状态进行微调。在应用于视觉三维导航任务时，我们的方法在重建准确性和任务性能上均实现了持续改进。进而研究表明，在部分可观测环境中考虑更多未来状态能进一步提升智能体性能，但在完全可观测环境中则无此效果。最后，我们证明预评估阶段训练较少的智能体从本方法中获益最大。