Video-language models (VLMs) achieve strong multimodal understanding but remain prone to hallucinations, especially when reasoning about actions and temporal order. Existing mitigation strategies, such as textual filtering or random video perturbations, often fail to address the root cause: over-reliance on language priors rather than fine-grained visual dynamics. We propose a scalable framework for counterfactual video generation that synthesizes videos differing only in actions or temporal structure while preserving scene context. Our pipeline combines multimodal LLMs for action proposal and editing guidance with diffusion-based image and video models to generate semantic hard negatives at scale. Using this framework, we build CounterVid, a synthetic dataset of ~26k preference pairs targeting action recognition and temporal reasoning. We further introduce MixDPO, a unified Direct Preference Optimization approach that jointly leverages textual and visual preferences. Fine-tuning Qwen2.5-VL with MixDPO yields consistent improvements, notably in temporal ordering, and transfers effectively to standard video hallucination benchmarks. Code and models will be made publicly available.

翻译：视频-语言模型（VLMs）在多模态理解方面表现出色，但在推理动作和时间顺序时仍容易产生幻觉。现有的缓解策略（如文本过滤或随机视频扰动）往往未能解决根本原因：过度依赖语言先验而非细粒度的视觉动态。我们提出了一种可扩展的反事实视频生成框架，能够合成仅在动作或时间结构上不同而保持场景上下文一致的视频。我们的流程结合了多模态大语言模型（用于动作提议与编辑指导）与基于扩散的图像和视频模型，从而大规模生成语义硬负例。利用该框架，我们构建了CounterVid——一个包含约26k个偏好对的合成数据集，专门针对动作识别与时间推理任务。我们进一步提出了MixDPO，一种统一的直接偏好优化方法，能够同时利用文本与视觉偏好。使用MixDPO对Qwen2.5-VL进行微调，在时间排序等任务上取得了显著且一致的性能提升，并能有效迁移至标准视频幻觉基准测试中。代码与模型将公开发布。