The LogSumExp function, dual to the Kullback-Leibler (KL) divergence, plays a central role in many important optimization problems, including entropy-regularized optimal transport (OT) and distributionally robust optimization (DRO). In practice, when the number of exponential terms inside the logarithm is large or infinite, optimization becomes challenging since computing the gradient requires differentiating every term. We propose a novel convexity- and smoothness-preserving approximation to LogSumExp that can be efficiently optimized using stochastic gradient methods. This approximation is rooted in a sound modification of the KL divergence in the dual, resulting in a new $f$-divergence called the safe KL divergence. Our experiments and theoretical analysis of the LogSumExp-based stochastic optimization, arising in DRO and continuous OT, demonstrate the advantages of our approach over existing baselines.

翻译：LogSumExp函数作为Kullback-Leibler（KL）散度的对偶形式，在诸多重要优化问题中扮演着核心角色，包括熵正则化最优传输（OT）与分布鲁棒优化（DRO）。在实际应用中，当对数内部指数项数量巨大或无穷时，由于梯度计算需要对每一项进行微分，优化过程变得极具挑战性。本文提出一种新颖的保持凸性与光滑性的LogSumExp近似方法，该方法可利用随机梯度方法进行高效优化。该近似源于对偶空间中KL散度的合理修正，从而产生一种称为安全KL散度的新型$f$-散度。我们在DRO与连续OT中基于LogSumExp的随机优化实验与理论分析表明，该方法相较于现有基线具有显著优势。