The connections between (convex) optimization and (logconcave) sampling have been considerably enriched in the past decade with many conceptual and mathematical analogies. For instance, the Langevin algorithm can be viewed as a sampling analogue of gradient descent and has condition-number-dependent guarantees on its performance. In the early 1990s, Nesterov and Nemirovski developed the Interior-Point Method (IPM) for convex optimization based on self-concordant barriers, providing efficient algorithms for structured convex optimization, often faster than the general method. This raises the following question: can we develop an analogous IPM for structured sampling problems? In 2012, Kannan and Narayanan proposed the Dikin walk for uniformly sampling polytopes, and an improved analysis was given in 2020 by Laddha-Lee-Vempala. The Dikin walk uses a local metric defined by a self-concordant barrier for linear constraints. Here we generalize this approach by developing and adapting IPM machinery together with the Dikin walk for poly-time sampling algorithms. Our IPM-based sampling framework provides an efficient warm start and goes beyond uniform distributions and linear constraints. We illustrate the approach on important special cases, in particular giving the fastest algorithms to sample uniform, exponential, or Gaussian distributions on a truncated PSD cone. The framework is general and can be applied to other sampling algorithms.
翻译:(凸)优化与(对数凹)采样之间的联系在过去十年中得到了显著丰富,并涌现出许多概念性和数学上的类比。例如,朗之万算法可视为梯度下降法的采样类比,其性能具有基于条件数的保证。20世纪90年代初,Nesterov和Nemirovski基于自调和障碍函数提出了凸优化的内点法(IPM),为结构化凸优化提供了高效算法,其速度通常优于通用方法。这引发了一个问题:我们能否为结构化采样问题开发类似的IPM?2012年,Kannan和Narayanan提出了用于多面体均匀采样的迪金漫步,而Laddha-Lee-Vempala于2020年给出了改进的分析。迪金漫步利用由线性约束的自调和障碍函数定义的局部度量。本文通过开发并适配IPM机制与迪金漫步,将这一方法推广至多项式时间采样算法。我们的基于IPM的采样框架提供了高效的冷启动,并超越了均匀分布与线性约束的范畴。我们通过重要特例展示了该方法的有效性,尤其是在截断半正定锥上实现了均匀分布、指数分布或高斯分布的最快采样算法。该框架具有通用性,可应用于其他采样算法。