We propose two novel extensions of the Wyner common information optimization problem. Each relaxes one fundamental constraints in Wyner's formulation. The \textit{Variational Wyner Common Information} relaxes the matching constraint to the known distribution while imposing conditional independence to the feasible solution set. We derive a tight surrogate upper bound of the obtained unconstrained Lagrangian via the theory of variational inference, which can be minimized efficiently. Our solver caters to problems where conditional independence holds with significantly reduced computation complexity; On the other hand, the \textit{Bipartite Wyner Common Information} relaxes the conditional independence constraint whereas the matching condition is enforced on the feasible set. By leveraging the difference-of-convex structure of the formulated optimization problem, we show that our solver is resilient to conditional dependent sources. Both solvers are provably convergent (local stationary points), and empirically, they obtain more accurate solutions to Wyner's formulation with substantially less runtime. Moreover, them can be extended to unknown distribution settings by parameterizing the common randomness as a member of the exponential family of distributions. Our approaches apply to multi-modal clustering problems, where multiple modalities of observations come from the same cluster. Empirically, our solvers outperform the state-of-the-art multi-modal clustering algorithms with significantly improved performance.

翻译：我们提出了Wyner公共信息优化问题的两种新扩展，分别放宽了Wyner原始表述中的一项基本约束。其中，变分Wyner公共信息（Variational Wyner Common Information）在保持可行解集条件独立性的同时，放宽了与已知分布匹配的约束。通过变分推断理论，我们推导出所得无约束拉格朗日函数的紧致代理上界，该上界可被高效优化。该求解器特别适用于条件独立性成立的场景，显著降低了计算复杂度；另一方面，二分Wyner公共信息（Bipartite Wyner Common Information）在可行解集强制匹配条件的同时，放宽了条件独立性约束。通过利用所构建优化问题的凸差结构，我们证明了该求解器对条件依赖信源具有鲁棒性。两个求解器均被证明可收敛至局部驻点，且实验表明，它们能以更短运行时间获得Wyner公式的更精确解。此外，通过将公共随机性参数化为指数族分布成员，可将其推广至未知分布场景。我们的方法适用于多模态聚类问题——该场景中同一聚类的观测数据包含多种模态。实验结果表明，所提求解器在性能上显著超越当前最优的多模态聚类算法。