We consider a general optimization problem of minimizing a composite objective functional defined over a class of probability distributions. The objective is composed of two functionals: one is assumed to possess the variational representation and the other is expressed in terms of the expectation operator of a possibly nonsmooth convex regularizer function. Such a regularized distributional optimization problem widely appears in machine learning and statistics, such as proximal Monte-Carlo sampling, Bayesian inference and generative modeling, for regularized estimation and generation. We propose a novel method, dubbed as Moreau-Yoshida Variational Transport (MYVT), for solving the regularized distributional optimization problem. First, as the name suggests, our method employs the Moreau-Yoshida envelope for a smooth approximation of the nonsmooth function in the objective. Second, we reformulate the approximate problem as a concave-convex saddle point problem by leveraging the variational representation, and then develope an efficient primal-dual algorithm to approximate the saddle point. Furthermore, we provide theoretical analyses and report experimental results to demonstrate the effectiveness of the proposed method.

翻译：本文考虑一类在概率分布上定义的最小化复合目标泛函的通用优化问题。该目标由两个泛函构成：一个假设具有变分表示，另一个通过可能非光滑凸正则化函数的期望算子表达。这类正则化分布优化问题广泛出现在机器学习和统计学中，例如近端蒙特卡洛采样、贝叶斯推断和生成建模中的正则化估计与生成任务。我们提出一种名为Moreau-Yoshida变分传输（MYVT）的新方法，用于求解正则化分布优化问题。首先，顾名思义，该方法采用Moreau-Yoshida包络对目标中非光滑函数进行光滑逼近。其次，我们利用变分表示将近似问题重构为凹-凸鞍点问题，并开发出高效的原始-对偶算法来逼近该鞍点。最后，我们通过理论分析和实验结果证明所提方法的有效性。