Non-Gaussian likelihoods are essential for modelling complex real-world observations but pose significant computational challenges in learning and inference. Even with Gaussian priors, non-Gaussian likelihoods often lead to analytically intractable posteriors, necessitating approximation methods. To this end, we propose efficient schemes to approximate the effects of non-Gaussian likelihoods by Gaussian densities based on variational inference and moment matching in transformed bases. These enable efficient inference strategies originally designed for models with a Gaussian likelihood to be deployed. Our empirical results demonstrate that the proposed matching strategies attain good approximation quality for binary and multiclass classification in large-scale point-estimate and distributional inferential settings. In challenging streaming problems, the proposed methods outperform all existing likelihood approximations and approximate inference methods in the exact models. As a by-product, we show that the proposed approximate log-likelihoods are a superior alternative to least-squares on raw labels for neural network classification.

翻译：非高斯似然对于建模复杂的现实世界观测数据至关重要，但在学习和推断过程中带来了显著的计算挑战。即使采用高斯先验，非高斯似然通常也会导致解析上难以处理的后验分布，从而需要近似方法。为此，我们提出了基于变分推断和变换基下矩匹配的高效方案，通过高斯密度来近似非高斯似然的影响。这使得原本为高斯似然模型设计的高效推断策略得以应用。我们的实证结果表明，所提出的匹配策略在大规模点估计和分布推断设置中，对二分类和多分类问题均能获得良好的近似质量。在具有挑战性的流式问题中，所提方法在精确模型上优于所有现有的似然近似和近似推断方法。作为副产品，我们证明了所提出的近似对数似然是神经网络分类中优于原始标签最小二乘法的替代方案。