The goal of this work is to understand the way actions are performed in videos. That is, given a video, we aim to predict an adverb indicating a modification applied to the action (e.g. cut "finely"). We cast this problem as a regression task. We measure textual relationships between verbs and adverbs to generate a regression target representing the action change we aim to learn. We test our approach on a range of datasets and achieve state-of-the-art results on both adverb prediction and antonym classification. Furthermore, we outperform previous work when we lift two commonly assumed conditions: the availability of action labels during testing and the pairing of adverbs as antonyms. Existing datasets for adverb recognition are either noisy, which makes learning difficult, or contain actions whose appearance is not influenced by adverbs, which makes evaluation less reliable. To address this, we collect a new high quality dataset: Adverbs in Recipes (AIR). We focus on instructional recipes videos, curating a set of actions that exhibit meaningful visual changes when performed differently. Videos in AIR are more tightly trimmed and were manually reviewed by multiple annotators to ensure high labelling quality. Results show that models learn better from AIR given its cleaner videos. At the same time, adverb prediction on AIR is challenging, demonstrating that there is considerable room for improvement.

翻译：本工作的目标是理解视频中行为的执行方式。即，给定一段视频，我们旨在预测一个指示行为修改方式的副词（例如"精细地"切）。我们将其建模为回归任务。通过测量动词与副词之间的文本关系，生成代表目标行为变化的回归目标。我们在多个数据集上测试该方法，并在副词预测和反义词分类任务上均取得最优结果。此外，我们放宽了测试时需提供行为标签和将副词配对为反义词这两个常见假设条件，性能仍优于先前工作。现有的副词识别数据集要么存在噪声导致学习困难，要么包含的行为外观不受副词影响导致评估不可靠。为此，我们构建了高质量新数据集：食谱中的副词（AIR）。聚焦于教学类食谱视频，精选一组在不同执行方式下呈现显著视觉差异的行为。AIR中的视频经过更精确的裁剪，并由多名标注者人工审核以确保标注质量。结果表明，由于AIR视频更清晰，模型能从中更有效地学习。同时，AIR上的副词预测任务具有挑战性，表明该领域仍有显著提升空间。