Video description entails automatically generating coherent natural language sentences that narrate the content of a given video. We introduce CLearViD, a transformer-based model for video description generation that leverages curriculum learning to accomplish this task. In particular, we investigate two curriculum strategies: (1) progressively exposing the model to more challenging samples by gradually applying a Gaussian noise to the video data, and (2) gradually reducing the capacity of the network through dropout during the training process. These methods enable the model to learn more robust and generalizable features. Moreover, CLearViD leverages the Mish activation function, which provides non-linearity and non-monotonicity and helps alleviate the issue of vanishing gradients. Our extensive experiments and ablation studies demonstrate the effectiveness of the proposed model. The results on two datasets, namely ActivityNet Captions and YouCook2, show that CLearViD significantly outperforms existing state-of-the-art models in terms of both accuracy and diversity metrics.

翻译：视频描述旨在自动生成连贯的自然语言句子，用以叙述给定视频的内容。我们提出CLearViD——一种基于Transformer的视频描述生成模型，通过引入课程学习来完成该任务。具体而言，我们研究了两种课程策略：（1）逐步对视频数据施加高斯噪声，使模型渐进式地接触更具挑战性的样本；（2）在训练过程中通过dropout逐步降低网络容量。这些方法使模型能够学习到更鲁棒且更具泛化能力的特征。此外，CLearViD采用Mish激活函数，该函数具有非线性和非单调特性，有助于缓解梯度消失问题。广泛的实验与消融研究证明了所提模型的有效性。在ActivityNet Captions和YouCook2两个数据集上的结果表明，CLearViD在准确性和多样性指标上均显著优于现有最优模型。