Open world Machine Learning (OWML) aims to develop intelligent systems capable of recognizing known categories, rejecting unknown samples, and continually learning from novel information. Despite significant progress in open set recognition, novelty detection, and continual learning, the field still lacks a unified theoretical foundation that can quantify uncertainty, characterize information transfer, and explain learning adaptability in dynamic, nonstationary environments. This paper presents a comprehensive review of information theoretic approaches in open world machine learning, emphasizing how core concepts such as entropy, mutual information, and Kullback Leibler divergence provide a mathematical language for describing knowledge acquisition, uncertainty suppression, and risk control under open world conditions. We synthesize recent studies into three major research axes: information theoretic open set recognition enabling safe rejection of unknowns, information driven novelty discovery guiding new concept formation, and information retentive continual learning ensuring stable long term adaptation. Furthermore, we discuss theoretical connections between information theory and provable learning frameworks, including PAC Bayes bounds, open-space risk theory, and causal information flow, to establish a pathway toward provable and trustworthy open world intelligence. Finally, the review identifies key open problems and future research directions, such as the quantification of information risk, development of dynamic mutual information bounds, multimodal information fusion, and integration of information theory with causal reasoning and world model learning.

翻译：开放世界机器学习（OWML）旨在开发能够识别已知类别、拒绝未知样本并持续从新信息中学习的智能系统。尽管在开放集识别、新颖性检测和持续学习方面取得了显著进展，该领域仍缺乏统一的理论基础，以量化不确定性、表征信息传递并解释动态非平稳环境中的学习适应性。本文全面综述了开放世界机器学习中的信息论方法，重点阐述熵、互信息和Kullback-Leibler散度等核心概念如何为描述开放世界条件下的知识获取、不确定性抑制和风险控制提供数学语言。我们将近期研究归纳为三大研究主轴：支持安全拒绝未知样本的信息论开放集识别、引导新概念形成的信息驱动新颖性发现，以及确保稳定长期适应的信息保留型持续学习。此外，我们探讨了信息论与可证明学习框架之间的理论联系，包括PAC-Bayes边界、开放空间风险理论和因果信息流，以建立通往可证明且可信赖的开放世界智能的路径。最后，本文指出了关键开放问题与未来研究方向，例如信息风险的量化、动态互信息边界的发展、多模态信息融合，以及信息论与因果推理及世界模型学习的整合。