Objects are crucial for understanding human-object interactions. By identifying the relevant objects, one can also predict potential future interactions or actions that may occur with these objects. In this paper, we study the problem of Short-Term Object interaction anticipation (STA) and propose NAOGAT (Next-Active-Object Guided Anticipation Transformer), a multi-modal end-to-end transformer network, that attends to objects in observed frames in order to anticipate the next-active-object (NAO) and, eventually, to guide the model to predict context-aware future actions. The task is challenging since it requires anticipating future action along with the object with which the action occurs and the time after which the interaction will begin, a.k.a. the time to contact (TTC). Compared to existing video modeling architectures for action anticipation, NAOGAT captures the relationship between objects and the global scene context in order to predict detections for the next active object and anticipate relevant future actions given these detections, leveraging the objects' dynamics to improve accuracy. One of the key strengths of our approach, in fact, is its ability to exploit the motion dynamics of objects within a given clip, which is often ignored by other models, and separately decoding the object-centric and motion-centric information. Through our experiments, we show that our model outperforms existing methods on two separate datasets, Ego4D and EpicKitchens-100 ("Unseen Set"), as measured by several additional metrics, such as time to contact, and next-active-object localization. The code will be available upon acceptance.

翻译：物体对理解人-物交互至关重要。通过识别相关物体，人们还能预测未来可能发生的交互或动作。本文研究短期物体交互预测（STA）问题，提出NAOGAT（下一活跃物体引导的预测Transformer）——一种多模态端到端Transformer网络，该网络关注已观测帧中的物体，以预测下一活跃物体（NAO），并最终引导模型预测情境感知的未来动作。该任务具有挑战性，因为需要同时预测未来动作、动作所涉及的物体以及交互开始前的剩余时间（即接触时间，TTC）。与现有用于动作预测的视频建模架构相比，NAOGAT通过捕捉物体与全局场景上下文之间的关系，预测下一活跃物体的检测结果，并基于这些检测结果预测相关的未来动作，利用物体动力学提升准确性。事实上，我们的方法关键优势之一在于能够利用给定片段中物体的运动动力学（这点常被其他模型忽视），并分别解码以物体为中心和以运动为中心的信息。实验表明，我们的模型在两个独立数据集（Ego4D和EpicKitchens-100的“未见集合”）上，通过多项附加指标（如接触时间、下一活跃物体定位）评估，均优于现有方法。代码将在论文被接收后公开。