Visual-tactile learning (VTL) enables embodied agents to perceive the physical world by integrating visual (VIS) and tactile (TAC) sensors. However, VTL still suffers from modality discrepancies between VIS and TAC images, as well as domain gaps caused by non-standardized tactile sensors and inconsistent data collection procedures. We formulate these challenges as a new task, termed single domain generalization for multimodal VTL (SDG-VTL). In this paper, we propose an OmniVaT framework that, for the first time, successfully addresses this task. On the one hand, OmniVaT integrates a multimodal fractional Fourier adapter (MFFA) to map VIS and TAC embeddings into a unified embedding-frequency space, thereby effectively mitigating the modality gap without multi-domain training data or careful cross-modal fusion strategies. On the other hand, it also incorporates a discrete tree generation (DTG) module that obtains diverse and reliable multimodal fractional representations through a hierarchical tree structure, thereby enhancing its adaptivity to fluctuating domain shifts in unseen domains. Extensive experiments demonstrate the superior cross-domain generalization performance of OmniVaT on the SDG-VTL task.

翻译：视觉-触觉学习（VTL）使具身智能体能够通过整合视觉（VIS）与触觉（TAC）传感器来感知物理世界。然而，VTL仍面临视觉与触觉图像间的模态差异，以及由非标准化触觉传感器和不一致数据采集流程导致的域间差距。我们将这些挑战形式化为一项新任务，即多模态VTL的单域泛化（SDG-VTL）。本文提出OmniVaT框架，首次成功应对该任务。一方面，OmniVaT集成多模态分数傅里叶适配器（MFFA），将VIS与TAC嵌入映射至统一的嵌入-频率空间，从而有效弥合模态间隙，且无需多域训练数据或精细的跨模态融合策略。另一方面，该框架还引入离散树生成（DTG）模块，通过分层树状结构获得多样且可靠的多模态分数表示，进而增强其对未见域中动态域偏移的适应能力。大量实验证明，OmniVaT在SDG-VTL任务上具有卓越的跨域泛化性能。