Recent works reveal that adversarial augmentation benefits the generalization of neural networks (NNs) if used in an appropriate manner. In this paper, we introduce Temporal Adversarial Augmentation (TA), a novel video augmentation technique that utilizes temporal attention. Unlike conventional adversarial augmentation, TA is specifically designed to shift the attention distributions of neural networks with respect to video clips by maximizing a temporal-related loss function. We demonstrate that TA will obtain diverse temporal views, which significantly affect the focus of neural networks. Training with these examples remedies the flaw of unbalanced temporal information perception and enhances the ability to defend against temporal shifts, ultimately leading to better generalization. To leverage TA, we propose Temporal Video Adversarial Fine-tuning (TAF) framework for improving video representations. TAF is a model-agnostic, generic, and interpretability-friendly training strategy. We evaluate TAF with four powerful models (TSM, GST, TAM, and TPN) over three challenging temporal-related benchmarks (Something-something V1&V2 and diving48). Experimental results demonstrate that TAF effectively improves the test accuracy of these models with notable margins without introducing additional parameters or computational costs. As a byproduct, TAF also improves the robustness under out-of-distribution (OOD) settings. Code is available at https://github.com/jinhaoduan/TAF.

翻译：近期研究表明，若以适当方式使用，对抗性增强能够提升神经网络（NNs）的泛化能力。本文提出一种新颖的视频增强技术——时间对抗性增强（TA），该技术利用时间注意力机制。与传统对抗性增强不同，TA通过最大化与时间相关的损失函数，专门设计用于调整神经网络对视频片段中的注意力分布。我们证明TA能够获得多样化的时间视图，显著影响神经网络的关注焦点。利用这些样本进行训练可弥补时间信息感知不平衡的缺陷，增强抵御时间偏移的能力，最终实现更好的泛化效果。为充分利用TA，我们提出时间视频对抗性微调（TAF）框架以改进视频表征。TAF是一种与模型无关、通用且可解释性友好的训练策略。我们在三个具有挑战性的时间相关基准数据集（Something-something V1&V2和diving48）上，使用四种强大的模型（TSM、GST、TAM和TPN）对TAF进行评估。实验结果表明，TAF在不引入额外参数或计算成本的情况下，显著提升了这些模型的测试准确率。作为附带成果，TAF还提高了在分布外（OOD）场景下的鲁棒性。代码已开源至https://github.com/jinhaoduan/TAF。