Video behavior recognition demands stable and discriminative representations under complex spatiotemporal variations. However, prevailing data augmentation strategies for videos remain largely perturbation-driven, often introducing uncontrolled variations that amplify non-discriminative factors, which finally weaken intra-class distributional structure and representation drift with inconsistent gains across temporal scales. To address these problems, we propose Representation-aware Mixing Augmentation (ReMA), a plug-and-play augmentation strategy that formulates mixing as a controlled replacement process to expand representations while preserving class-conditional stability. ReMA integrates two complementary mechanisms. Firstly, the Representation Alignment Mechanism (RAM) performs structured intra-class mixing under distributional alignment constraints, suppressing irrelevant intra-class drift while enhancing statistical reliability. Then, the Dynamic Selection Mechanism (DSM) generates motion-aware spatiotemporal masks to localize perturbations, guiding them away from discrimination-sensitive regions and promoting temporal coherence. By jointly controlling how and where mixing is applied, ReMA improves representation robustness without additional supervision or trainable parameters. Extensive experiments on diverse video behavior benchmarks demonstrate that ReMA consistently enhances generalization and robustness across different spatiotemporal granularities.

翻译：视频行为识别需要在复杂的时空变化下获得稳定且具有判别性的表征。然而，当前主流的视频数据增强策略在很大程度上仍是扰动驱动的，通常会引入不受控的变化，从而放大非判别性因素，最终削弱类内分布结构并导致表征漂移，且在时间尺度上带来不一致的收益。为解决这些问题，我们提出了表征感知混合增强（ReMA），一种即插即用的增强策略，它将混合过程形式化为一种受控的替换过程，以在保持类条件稳定性的同时扩展表征。ReMA整合了两种互补机制。首先，表征对齐机制（RAM）在分布对齐约束下执行结构化的类内混合，抑制无关的类内漂移同时增强统计可靠性。其次，动态选择机制（DSM）生成运动感知的时空掩码来定位扰动，引导其远离判别敏感区域并促进时间一致性。通过联合控制混合的“方式”与“位置”，ReMA无需额外监督或可训练参数即可提升表征的鲁棒性。在多种视频行为基准数据集上的大量实验表明，ReMA能持续提升模型在不同时空粒度下的泛化能力和鲁棒性。