Recent works reveal that adversarial augmentation benefits the generalization of neural networks (NNs) if used in an appropriate manner. In this paper, we introduce Temporal Adversarial Augmentation (TA), a novel video augmentation technique that utilizes temporal attention. Unlike conventional adversarial augmentation, TA is specifically designed to shift the attention distributions of neural networks with respect to video clips by maximizing a temporal-related loss function. We demonstrate that TA will obtain diverse temporal views, which significantly affect the focus of neural networks. Training with these examples remedies the flaw of unbalanced temporal information perception and enhances the ability to defend against temporal shifts, ultimately leading to better generalization. To leverage TA, we propose Temporal Video Adversarial Fine-tuning (TAF) framework for improving video representations. TAF is a model-agnostic, generic, and interpretability-friendly training strategy. We evaluate TAF with four powerful models (TSM, GST, TAM, and TPN) over three challenging temporal-related benchmarks (Something-something V1&V2 and diving48). Experimental results demonstrate that TAF effectively improves the test accuracy of these models with notable margins without introducing additional parameters or computational costs. As a byproduct, TAF also improves the robustness under out-of-distribution (OOD) settings. Code is available at https://github.com/jinhaoduan/TAF.

翻译：近期研究表明，若以适当方式使用，对抗性增强可提升神经网络（NNs）的泛化能力。本文提出一种新颖的视频增强技术——时间对抗性增强（TA），其利用时间注意力机制。与传统对抗性增强不同，TA通过最大化与时间相关的损失函数，专门设计用于改变神经网络对视频片段的注意力分布。我们证明TA能获得多样化的时间视角，显著影响神经网络关注的焦点。利用这些样本进行训练可弥补时间信息感知不均衡的缺陷，并增强抵御时间偏移的能力，最终实现更好的泛化效果。为充分利用TA，我们提出时间视频对抗性微调（TAF）框架以改进视频表示。TAF是一种模型无关、通用且易于解释的训练策略。我们基于三个具有挑战性的时间相关基准（Something-something V1&V2及diving48），使用四种强力模型（TSM、GST、TAM和TPN）对TAF进行评估。实验结果表明，TAF在不引入额外参数或计算成本的情况下，显著提升了这些模型的测试准确率。作为副产品，TAF还提升了模型在分布外（OOD）设置下的鲁棒性。代码已开源至https://github.com/jinhaoduan/TAF。