Detecting and responding to novel situations in open-world environments is a key capability of human cognition. Current artificial intelligence (AI) researchers strive to develop systems that can perform in open-world environments. Novelty detection is an important ability of such AI systems. In an open-world, novelties appear in various forms and the difficulty to detect them varies. Therefore, to accurately evaluate the detection capability of AI systems, it is necessary to investigate the difficulty to detect novelties. In this paper, we propose a qualitative physics-based method to quantify the difficulty of novelty detection focusing on open-world physical domains. We apply our method in a popular physics simulation game, Angry Birds. We conduct an experiment with human players with different novelties in Angry Birds to validate our method. Results indicate that the calculated difficulty values are in line with the detection difficulty of the human players.

翻译：发现和应对开放世界环境中的新情况是人类认知的关键能力。当前人工智能(AI)研究人员努力开发能够在开放世界环境中运行的系统。新发现是这种人工智能系统的一个重要能力。在开放世界中,新发现以各种形式出现,发现新情况的困难也各有不同。因此,为了准确评估人工智能系统的检测能力,有必要调查发现新情况的困难。在本文中,我们建议一种基于物理的定性方法,以量化以开放世界物理领域为重点的新发现困难。我们在流行的物理模拟游戏中应用我们的方法,愤怒鸟。我们与愤怒鸟中不同新手的人类玩家进行了实验,以验证我们的方法。结果显示计算的困难值与人类玩家的检测困难值相符。