The principle of maximum entropy, as introduced by Jaynes in information theory, has contributed to advancements in various domains such as Statistical Mechanics, Machine Learning, and Ecology. Its resultant solutions have served as a catalyst, facilitating researchers in mapping their empirical observations to the acquisition of unbiased models, whilst deepening the understanding of complex systems and phenomena. However, when we consider situations in which the model elements are not directly observable, such as when noise or ocular occlusion is present, possibilities arise for which standard maximum entropy approaches may fail, as they are unable to match feature constraints. Here we show the Principle of Uncertain Maximum Entropy as a method that both encodes all available information in spite of arbitrarily noisy observations while surpassing the accuracy of some ad-hoc methods. Additionally, we utilize the output of a black-box machine learning model as input into an uncertain maximum entropy model, resulting in a novel approach for scenarios where the observation function is unavailable. Previous remedies either relaxed feature constraints when accounting for observation error, given well-characterized errors such as zero-mean Gaussian, or chose to simply select the most likely model element given an observation. We anticipate our principle finding broad applications in diverse fields due to generalizing the traditional maximum entropy method with the ability to utilize uncertain observations.

翻译：最大熵原理由杰恩斯在信息论中提出，已推动统计力学、机器学习及生态学等多个领域的发展。其解作为催化剂，帮助研究者将经验观测映射为无偏模型，同时深化对复杂系统与现象的理解。然而，当模型元素无法直接观测（例如存在噪声或视觉遮挡）时，标准最大熵方法可能因无法匹配特征约束而失效。本文提出不确定最大熵原理，该方法能够在任意噪声观测下编码所有可用信息，同时超越若干特设方法的准确性。此外，我们将黑箱机器学习模型的输出作为不确定最大熵模型的输入，为观测函数不可用场景提供了新方法。以往针对观测误差（如零均值高斯噪声）的补救措施或放宽特征约束，或直接选择给定观测下最可能的模型元素。我们预计该原理因传统最大熵方法的泛化及利用不确定观测的能力，将在不同领域获得广泛应用。