Abductive reasoning aims to make the most likely inference for a given set of incomplete observations. In this work, we propose a new task called abductive action inference, in which given a situation, the model answers the question `what actions were executed by the human in order to arrive in the current state?'. Given a state, we investigate three abductive inference problems: action set prediction, action sequence prediction, and abductive action verification. We benchmark several SOTA models such as Transformers, Graph neural networks, CLIP, BLIP, end-to-end trained Slow-Fast, and Resnet50-3D models. Our newly proposed object-relational BiGED model outperforms all other methods on this challenging task on the Action Genome dataset. Codes will be made available.

翻译：溯因推理旨在针对给定的一组不完整观测数据做出最可能的推断。本文提出了一项名为溯因行为推理的新任务，即给定某个情境，模型需回答“人类为到达当前状态执行了哪些行为？”这一问题。针对某个状态，我们研究了三种溯因推理问题：行为集合预测、行为序列预测以及溯因行为验证。我们对多种先进模型进行了基准测试，包括Transformer、图神经网络、CLIP、BLIP、端到端训练的Slow-Fast以及Resnet50-3D模型。我们新提出的对象关系型BiGED模型在Action Genome数据集上的这一具有挑战性的任务中优于所有其他方法。代码将公开提供。