In this paper we present a practical Bayesian self-supervised learning method with Cyclical Stochastic Gradient Hamiltonian Monte Carlo (cSGHMC). Within this framework, we place a prior over the parameters of a self-supervised learning model and use cSGHMC to approximate the high dimensional and multimodal posterior distribution over the embeddings. By exploring an expressive posterior over the embeddings, Bayesian self-supervised learning produces interpretable and diverse representations. Marginalizing over these representations yields a significant gain in performance, calibration and out-of-distribution detection on a variety of downstream classification tasks. We provide experimental results on multiple classification tasks on four challenging datasets. Moreover, we demonstrate the effectiveness of the proposed method in out-of-distribution detection using the SVHN and CIFAR-10 datasets.

翻译：本文提出了一种实用的贝叶斯自监督学习方法，该方法采用循环随机梯度哈密顿蒙特卡洛（cSGHMC）。在该框架中，我们在自监督学习模型的参数上引入先验分布，并利用cSGHMC来逼近嵌入空间的高维多模态后验分布。通过探索嵌入空间中的表达性后验分布，贝叶斯自监督学习能够产生可解释且多样化的表示。对这些表示进行边缘化处理，可在多种下游分类任务中显著提升性能、校准效果以及分布外检测能力。我们在四个具有挑战性的数据集上进行了多项分类任务的实验验证。此外，我们通过在SVHN和CIFAR-10数据集上的实验，证明了所提方法在分布外检测中的有效性。