Video-Action Models (VAMs) have emerged as a promising framework for embodied intelligence, learning implicit world dynamics from raw video streams to produce temporally consistent action predictions. Although such models demonstrate strong performance on long-horizon tasks through visual reasoning, they remain limited in contact-rich scenarios where critical interaction states are only partially observable from vision alone. In particular, fine-grained force modulation and contact transitions are not reliably encoded in visual tokens, leading to unstable or imprecise behaviors. To bridge this gap, we introduce the Video-Tactile Action Model (VTAM), a multimodal world modeling framework that incorporates tactile perception as a complementary grounding signal. VTAM augments a pretrained video transformer with tactile streams via a lightweight modality transfer finetuning, enabling efficient cross-modal representation learning without tactile-language paired data or independent tactile pretraining. To stabilize multimodal fusion, we introduce a tactile regularization loss that enforces balanced cross-modal attention, preventing visual latent dominance in the action model. VTAM demonstrates superior performance in contact-rich manipulation, maintaining a robust success rate of 90 percent on average. In challenging scenarios such as potato chip pick-and-place requiring high-fidelity force awareness, VTAM outperforms the pi 0.5 baseline by 80 percent. Our findings demonstrate that integrating tactile feedback is essential for correcting visual estimation errors in world action models, providing a scalable approach to physically grounded embodied foundation models.

翻译：视频-动作模型（VAMs）已成为具身智能领域颇具前景的框架，能够从原始视频流中学习隐式世界动态，以生成时序一致的动作预测。尽管此类模型通过视觉推理在长期任务中展现出强大性能，但在关键交互状态仅能通过视觉部分观测的接触密集型场景中仍存在局限。具体而言，细粒度力调控与接触状态的转换无法被视觉令牌可靠编码，导致行为不稳定或不精确。为弥合这一差距，我们提出视频-触觉-动作模型（VTAM），这是一种将触觉感知作为互补接地信号的多模态世界建模框架。通过轻量级模态迁移微调，VTAM利用触觉流增强预训练视频Transformer，无需触觉-语言配对数据或独立触觉预训练即可实现高效跨模态表征学习。为稳定多模态融合，我们引入触觉正则化损失函数，强制实现均衡的跨模态注意力机制，从而避免动作模型中视觉潜在主导的倾向。VTAM在接触密集型操作任务中表现卓越，平均成功率达90%。在诸如需要高保真力感知的薯片拾取-放置等挑战性场景中，VTAM比pi 0.5基线模型性能提升80%。我们的研究证明，将触觉反馈集成到世界动作模型中对修正视觉估计误差至关重要，为物理接地具身基础模型提供了可扩展方案。