We study Compositional Video Understanding (CVU), where models must recognize verbs and objects and compose them to generalize to unseen combinations. We find that existing Zero-Shot Compositional Action Recognition (ZS-CAR) models fail primarily due to an overlooked failure mode: object-driven verb shortcuts. Through systematic analysis, we show that this behavior arises from two intertwined factors: severe sparsity and skewness of compositional supervision, and the asymmetric learning difficulty between verbs and objects. As training progresses, the existing ZS-CAR model increasingly ignores visual evidence and overfits to co-occurrence statistics. Consequently, the existing model does not gain the benefit of compositional recognition in unseen verb-object compositions. To address this, we propose RCORE, a simple and effective framework that enforces temporally grounded verb learning. RCORE introduces (i) a composition-aware augmentation that diversifies verb-object combinations without corrupting motion cues, and (ii) a temporal order regularization loss that penalizes shortcut behaviors by explicitly modeling temporal structure. Across two benchmarks, Sth-com and our newly constructed EK100-com, RCORE significantly improves unseen composition accuracy, reduces reliance on co-occurrence bias, and achieves consistently positive compositional gaps. Our findings reveal object-driven shortcuts as a critical limiting factor in ZS-CAR and demonstrate that addressing them is essential for robust compositional video understanding.

翻译：我们研究组合视频理解（CVU），其中模型必须识别动词和物体并将其组合以泛化至未见过的组合。我们发现现有的零样本组合动作识别（ZS-CAR）模型失败的主要原因是未被重视的失效模式：物体驱动的动词捷径。通过系统分析，我们证明该行为源于两个相互交织的因素：组合监督的严重稀疏性与偏态性，以及动词与物体间非对称的学习难度。随着训练进行，现有ZS-CAR模型逐渐忽略视觉证据并过度拟合共现统计。因此，现有模型无法在未见过的动词-物体组合中获得组合识别的优势。为解决此问题，我们提出RCORE——一个简单有效的框架，通过强制实施时序锚定的动词学习。RCORE引入（i）组合感知的数据增强，在不破坏运动线索的前提下多样化动词-物体组合；（ii）时序顺序正则化损失，通过显式建模时序结构来惩罚捷径行为。在两个基准测试（Sth-com与我们新构建的EK100-com）中，RCORE显著提升了未见组合的准确率，降低了对共现偏差的依赖，并实现了持续为正的组合泛化差距。我们的研究揭示了物体驱动捷径是ZS-CAR的关键限制因素，并证明解决该问题对于实现稳健的组合视频理解至关重要。