The principle of maximum entropy, as introduced by Jaynes in information theory, has contributed to advancements in various domains such as Statistical Mechanics, Machine Learning, and Ecology. Its resultant solutions have served as a catalyst, facilitating researchers in mapping their empirical observations to the acquisition of unbiased models, whilst deepening the understanding of complex systems and phenomena. However, when we consider situations in which the model elements are not directly observable, such as when noise or ocular occlusion is present, possibilities arise for which standard maximum entropy approaches may fail, as they are unable to match feature constraints. Here we show the Principle of Uncertain Maximum Entropy as a method that both encodes all available information in spite of arbitrarily noisy observations while surpassing the accuracy of some ad-hoc methods. Additionally, we utilize the output of a black-box machine learning model as input into an uncertain maximum entropy model, resulting in a novel approach for scenarios where the observation function is unavailable. Previous remedies either relaxed feature constraints when accounting for observation error, given well-characterized errors such as zero-mean Gaussian, or chose to simply select the most likely model element given an observation. We anticipate our principle finding broad applications in diverse fields due to generalizing the traditional maximum entropy method with the ability to utilize uncertain observations.

翻译：杰恩斯在信息论中提出的最大熵原理，推动了统计力学、机器学习和生态学等多个领域的发展。其求解结果作为催化剂，帮助研究者将经验观测映射到无偏模型的获取中，同时加深了对复杂系统与现象的理解。然而，当模型元素无法直接观测时（例如存在噪声或视觉遮挡），标准最大熵方法可能因无法匹配特征约束而失效。本文提出不确定最大熵原理：该方法既能在任意噪声观测下编码所有可用信息，又能超越若干特定方法的精度。此外，我们将黑箱机器学习模型的输出作为不确定最大熵模型的输入，为观测函数不可用的场景提供了新方案。先前的补救措施要么在考虑观测误差时放松特征约束（如针对零均值高斯噪声等特征明确的误差），要么直接根据观测选择最可能的模型元素。我们预计，由于该原理通过利用不确定观测推广了传统最大熵方法，将在多个领域获得广泛应用。