The Bayesian Learning Rule provides a framework for generic algorithm design but can be difficult to use for three reasons. First, it requires a specific parameterization of exponential family. Second, it uses gradients which can be difficult to compute. Third, its update may not always stay on the manifold. We address these difficulties by proposing an extension based on Lie-groups where posteriors are parametrized through transformations of an arbitrary base distribution and updated via the group's exponential map. This simplifies all three difficulties for many cases, providing flexible parametrizations through group's action, simple gradient computation through reparameterization, and updates that always stay on the manifold. We use the new learning rule to derive a new algorithm for deep learning with desirable biologically-plausible attributes to learn sparse features. Our work opens a new frontier for the design of new algorithms by exploiting Lie-group structures.

翻译：贝叶斯学习法则为通用算法设计提供了框架，但因其存在三个难点而难以应用：其一，需要指数族的特定参数化形式；其二，需计算梯度但往往难以实现；其三，算法更新可能偏离参数流形。为解决这些问题，我们提出基于李群的扩展方法：通过任意基分布的变换对后验分布进行参数化，并借助群的指数映射实施更新。该方案在三方面降低了原法则的运用难度：利用群作用实现灵活的参数化表示，通过重参数化简化梯度计算，且更新过程始终保持在流形上。基于此新学习法则，我们推导出适用于深度学习的新算法，该算法具备理想的生物合理性特征，可有效学习稀疏特征。本工作通过挖掘李群结构，为设计新型算法开辟了新方向。