Although action recognition for procedural tasks has received notable attention, it has a fundamental flaw in that no measure of success for actions is provided. This limits the applicability of such systems especially within the industrial domain, since the outcome of procedural actions is often significantly more important than the mere execution. To address this limitation, we define the novel task of procedure step recognition (PSR), focusing on recognizing the correct completion and order of procedural steps. Alongside the new task, we also present the multi-modal IndustReal dataset. Unlike currently available datasets, IndustReal contains procedural errors (such as omissions) as well as execution errors. A significant part of these errors are exclusively present in the validation and test sets, making IndustReal suitable to evaluate robustness of algorithms to new, unseen mistakes. Additionally, to encourage reproducibility and allow for scalable approaches trained on synthetic data, the 3D models of all parts are publicly available. Annotations and benchmark performance are provided for action recognition and assembly state detection, as well as the new PSR task. IndustReal, along with the code and model weights, is available at: https://github.com/TimSchoonbeek/IndustReal .

翻译：摘要：尽管程序性任务的动作识别已受到显著关注，但其存在一个根本缺陷——缺乏对动作成功程度的衡量。这限制了此类系统在工业领域的适用性，因为程序性动作的结果往往比单纯的动作执行重要得多。为解决此局限，我们定义了名为程序步骤识别（PSR）的新任务，重点识别程序步骤的正确完成情况与执行顺序。伴随新任务，我们同时提出多模态数据集IndustReal。与现有数据集不同，IndustReal包含程序性错误（如遗漏）及执行错误。这些错误的重要组成部分仅出现在验证集和测试集中，使IndustReal适用于评估算法对未见新错误的鲁棒性。此外，为促进可重复性并支持基于合成数据的可扩展方法，所有零件的3D模型均公开提供。我们针对动作识别、装配状态检测及新提出的PSR任务提供了标注和基线性能。IndustReal及其代码、模型权重可通过https://github.com/TimSchoonbeek/IndustReal获取。