We propose a general way to integrate procedural knowledge of a domain into deep learning models. We apply it to the case of video prediction, building on top of object-centric deep models and show that this leads to a better performance than using data-driven models alone. We develop an architecture that facilitates latent space disentanglement in order to use the integrated procedural knowledge, and establish a setup that allows the model to learn the procedural interface in the latent space using the downstream task of video prediction. We contrast the performance to a state-of-the-art data-driven approach and show that problems where purely data-driven approaches struggle can be handled by using knowledge about the domain, providing an alternative to simply collecting more data.

翻译：我们提出了一种将领域过程知识整合到深度学习模型中的通用方法。我们将其应用于视频预测任务，在基于对象中心的深度模型基础上进行构建，并证明该方法比单纯使用数据驱动模型能获得更好的性能。我们开发了一种促进潜在空间解缠结的架构，以便利用所整合的过程知识，并建立了一个允许模型通过视频预测这一下游任务来学习潜在空间中过程接口的设置。我们将该方法的性能与最先进的数据驱动方法进行了对比，结果表明：对于纯数据驱动方法难以处理的问题，通过利用领域知识可以有效地解决，这为单纯收集更多数据提供了一种替代方案。