Current AI systems are designed to solve close-world problems with the assumption that the underlying world is remaining more or less the same. However, when dealing with real-world problems such assumptions can be invalid as sudden and unexpected changes can occur. To effectively deploy AI-powered systems in the real world, AI systems should be able to deal with open-world novelty quickly. Inevitably, dealing with open-world novelty raises an important question of novelty difficulty. Knowing whether one novelty is harder to deal with than another, can help researchers to train their systems systematically. In addition, it can also serve as a measurement of the performance of novelty robust AI systems. In this paper, we propose to define the novelty reaction difficulty as a relative difficulty of performing the known task after the introduction of the novelty. We propose a universal method that can be applied to approximate the difficulty. We present the approximations of the difficulty using our method and show how it aligns with the results of the evaluation of AI agents designed to deal with novelty.

翻译：目前的AI系统旨在解决近乎世界的问题,其假设是,基础世界基本上保持不变。然而,当处理现实世界的问题时,这种假设可能无效,因为可能发生突然和意外的变化。为了在现实世界中有效地部署AI动力系统,AI系统应当能够迅速处理开放世界的新颖现象。处理开放世界的新颖现象不可避免地会产生一个重要的新困难问题。了解一个新事物是否比另一个更难处理,可以帮助研究人员系统地培训其系统。此外,它也可以作为衡量新颖的强大AI系统绩效的尺度。在本文件中,我们提议将新颖反应困难界定为在引进新事物后执行已知任务的相对困难。我们建议一种通用方法,可以用来弥补困难。我们用我们的方法来介绍困难的近似值,并表明它如何与旨在处理新事物的AI代理的评价结果相一致。