The constrained minimization (respectively maximization) of directed distances and of related generalized entropies is a fundamental task in information theory as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition. In our previous paper "A precise bare simulation approach to the minimization of some distances. I. Foundations", we obtained such kind of constrained optima by a new dimension-free precise bare (pure) simulation method, provided basically that (i) the underlying directed distance is of f-divergence type, and that (ii) this can be connected to a light-tailed probability distribution in a certain manner. In the present paper, we extend this approach such that constrained optimization problems of a very huge amount of directed distances and generalized entropies -- and beyond -- can be tackled by a newly developed dimension-free extended bare simulation method, for obtaining both optima as well as optimizers. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). For instance, we cover constrained optimizations of arbitrary f-divergences, Bregman distances, scaled Bregman distances and weighted Euclidean distances. The potential for wide-spread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences in various different research fields (which may also serve as an interdisciplinary interface).
翻译:有向距离及相关广义熵的约束最小化(分别对应最大化)是信息论以及统计学、机器学习、人工智能、信号处理和模式识别等相邻领域的基础任务。在我们之前的论文《一种精确的裸模拟方法用于若干距离的最小化. I. 基础》中,我们通过一种新的、无维度限制的精确裸(纯)模拟方法获得了此类约束最优解,基本前提是:(i)底层有向距离属于f-散度类型,且(ii)该距离能以特定方式与轻尾概率分布相关联。在本文中,我们扩展了这一方法,使得大量有向距离和广义熵(以及更多)的约束优化问题可以通过新开发的无维度限制的扩展裸模拟方法得到处理,从而同时获得最优值及优化子。在我们的任意维度离散设定下,几乎不需要对约束集做任何假设(如凸性),且我们的方法具有精确性(即在极限意义上收敛)。例如,我们涵盖了任意f-散度、Bregman距离、缩放后的Bregman距离和加权欧氏距离的约束优化。文中还指出了该方法广泛应用的潜力;特别是,我们提供了大量近期参考文献,涉及所涉及的距离/散度在不同研究领域中的应用(这些参考文献也可作为跨学科的接口)。