Vision-language-action (VLA) models typically rely on large-scale real-world videos, whereas simulated data, despite being inexpensive and highly parallelizable to collect, often suffers from a substantial visual domain gap and limited environmental diversity, resulting in weak real-world generalization. We present an efficient video augmentation framework that converts simulated VLA videos into realistic training videos while preserving task semantics and action trajectories. Our pipeline extracts structured conditions from simulation via video semantic segmentation and video captioning, rewrites captions to diversify environments, and uses a conditional video transfer model to synthesize realistic videos. To make augmentation practical at scale, we introduce a diffusion feature-reuse mechanism that reuses video tokens across adjacent timesteps to accelerate generation, and a coreset sampling strategy that identifies a compact, non-redundant subset for augmentation under limited computation. Extensive experiments on Robotwin 2.0, LIBERO, LIBERO-Plus, and a real robotic platform demonstrate consistent improvements. For example, our method improves RDT-1B by 8% on Robotwin 2.0, and boosts $π_0$ by 5.1% on the more challenging LIBERO-Plus benchmark. Code is available at: https://github.com/nanfangxiansheng/Seeing-Realism-from-Simulation.
翻译:视觉-语言-动作(VLA)模型通常依赖大规模真实世界视频,而仿真数据尽管成本低廉且易于并行采集,却常因显著的视觉域差异和环境多样性不足导致真实世界泛化能力薄弱。我们提出一种高效视频增强框架,可将仿真VLA视频转换为逼真的训练视频,同时保留任务语义和动作轨迹。该流程通过视频语义分割和视频描述从仿真中提取结构化条件,改写描述以丰富环境多样性,并利用条件视频迁移模型合成逼真视频。为提升增强的规模化实用性,我们引入扩散特征复用机制,通过跨相邻时间步复用视频令牌加速生成;同时提出核心集采样策略,在有限计算资源下识别紧凑无冗余的子集进行增强。在Robotwin 2.0、LIBERO、LIBERO-Plus及真实机器人平台上的大量实验均表明性能持续提升。例如,本方法在Robotwin 2.0上使RDT-1B提升8%,在更具挑战性的LIBERO-Plus基准上将π₀提升5.1%。代码开源地址:https://github.com/nanfangxiansheng/Seeing-Realism-from-Simulation