Estimating the state of an environment from high-dimensional, multimodal, and noisy observations is a fundamental challenge in reinforcement learning (RL). Traditional approaches rely on probabilistic models to account for the uncertainty, but often require explicit noise assumptions, in turn limiting generalization. In this work, we contribute a novel method to learn a structured latent representation, in which distances between states directly correlate with the minimum number of actions required to transition between them. The proposed metric space formulation provides a geometric interpretation of uncertainty without the need for explicit probabilistic modeling. To achieve this, we introduce a multimodal latent transition model and a sensor fusion mechanism based on inverse distance weighting, allowing for the adaptive integration of multiple sensor modalities without prior knowledge of noise distributions. We empirically validate the approach on a range of multimodal RL tasks, demonstrating improved robustness to sensor noise and superior state estimation compared to baseline methods. Our experiments show enhanced performance of an RL agent via the learned representation, eliminating the need of explicit noise augmentation. The presented results suggest that leveraging transition-aware metric spaces provides a principled and scalable solution for robust state estimation in sequential decision-making.

翻译：从高维、多模态且含噪声的观测中估计环境状态是强化学习（RL）中的一个基础性挑战。传统方法依赖概率模型来处理不确定性，但通常需要明确的噪声假设，从而限制了泛化能力。本文提出一种新颖的方法来学习结构化潜在表示，其中状态之间的距离直接关联于状态间转换所需的最小动作数。所提出的度量空间公式为不确定性提供了一种几何解释，无需显式的概率建模。为实现这一目标，我们引入了一个多模态潜在转移模型和一个基于反距离加权的传感器融合机制，允许在无需先验噪声分布知识的情况下自适应地整合多个传感器模态。我们在一系列多模态RL任务上对所提方法进行了实证验证，结果表明相较于基线方法，该方法对传感器噪声具有更强的鲁棒性，且状态估计性能更优。实验表明，通过习得的表示，RL智能体的性能得到提升，且无需显式的噪声增强。所呈现的结果表明，利用转移感知的度量空间为序列决策中的鲁棒状态估计提供了一个原则性且可扩展的解决方案。